Showing posts with label flac. Show all posts
Showing posts with label flac. Show all posts

Thursday, October 14, 2010

PowerShell GCI (Get-ChildItem) LiteralPath Issue

Test Music Directory for Powershell Scripts
After finishing ripping our CD collection to FLAC format (link) we wrote some PowerShell scripts to analyze the directory hierarchy that the FLAC (*.flac) files reside in. The hierarchy is something like this \\server\music\artist\album\track.flac where “artist”, “album”, and “track” change according the CD and track ripped. Starting at the top of the node \\server\music the scripts generate the following stats:

1. Number of album folders that do not have music (at least one .flac files). This can indicate that something went wrong and needs to be checked.

2. Number of artist folders that have no album folders in them. For housekeeping reasons it’s not necessary to have empty folders hanging around.

3. Number of .flac files that are less than 500 kb. This can indicate a potential problem with the ripping process. Some .flac files can be that small such as hidden files or silent tracks.

In the course of ripping CDs the following situation for 2+ disc collection can occur:

\\server\music\artistBlah\albumBlah [Disc 1]
\\server\music\artistBlah\albumBlah [Disc 2]

where the “[Disc 1]” and “[Disc 2]” are part of the folder name.

So we have our directory structure defined, what are the basics of the PowerShell scripts? They are .ps1 files that run at a PowerShell command prompt. The key command used in the scripts is the Get-ChildItem cmdlet, or commonly typed as just gci. The gci cmdlet operates like a directory listing. The gci cmdlet has several switches that help refine what you want to have returned like only files with certain file extensions (*.flac). The scripts work by first invoking a gci (with appropriate switches) at the root directory (\\server\music), capturing that into an array, and then iterating through that array and checking each array entry for specific criteria. The problem we ran into is that some switches of the gci cmdlet did not work well with the use of brackets “[“ and “]” in the folder name.

To make the problem we encountered more concrete, let’s suppose we have the following directory structure:

..\artist1\album1 [disc 1]
file.flac
folder.jpg
..\artist1\album1 [disc 2]
file.flac
..\artist1\album2
file.flac

In a PowerShell command window if you invoke the following command:
gci -path ("\\server\music\artist1\album1 [disc 1]") -recurse -include *.flac

The result is that nothing is found when we are expecting that is should return the file.flac.

Similarly, if you invoke the following command:
gci -path ("\\server\music\artist1\album2") -recurse -include *.flac

The result returns file.flac as expected.

The problem with the first command is the use of brackets in the first path (folder name) which are perfectly legal to use in folder names but PowerShell interprets these as range operators as discussed in this TechNet Tip. To get around the problem of having PowerShell interpret the range character we can use the -literalpath switch of the gci cmdlet.

So invoking this command:
gci -literalpath ("\\server\music\artist1\album 1 [disc 1]")

Returns file.flac as expected. The problem of using –literalpath intead of –path is that the –include switch seems to be ignored so we can’t refine what we are looking for. One workaround for this – and the point of this post – is that you have to then iterate through the items in each album folder. The following code snippet shows a workaround that will work:

$assetList = gci -literalpath ("C:\temp\music\artist1\album 1 [disc 1]")
if ($assetList -ne $null)
{
foreach ($item in $assetList )
{
$n = $item.Name.ToLower();
if ($n.EndsWith(".flac"))
{
#Do something to indicate a match was found.
}
}
}

For more information on this problem see this TechNet Tip and this forum post. Here’s a simple PowerShell script for iterating through a two level directory structure looking for *.flac files.


#
# usage : .\findDirectoriesWithNoMusic.ps1 "root"
# example: .\findDirectoriesWithNoMusic.ps1 "\\server\music"
# output : report.txt
#

$arg = $args[0]
if ($arg.Length -lt 1)
{
write-host "****************************************************
No root directory specified...
usage: $output = .\findDirectoriesWithNoMusic.ps1 root
****************************************************"
exit
}

Write-Progress "Getting directory information" -status "This could take several minutes for a large number of directories."
$list = gci -path ($arg) -recurse -exclude *.* # the first use of GCI

$output = "Report of directories with no music `n"

for ($i=0; $i -le $list.Length - 1; $i++)
{
if ($list[$i].Parent.FullName -ne $arg)
{
$albumPath = ($list[$i].Parent.Name + "\" + $list[$i].Name).ToLower()
$assetList = gci -literalpath ($arg + "\" + $albumPath) # the second use of GCI because we can't be sure if brackets [] exist in the $albumPath
$flag = 0;
if ($assetList -ne $null)
{
foreach ($item in $assetList )
{
$n = $item.Name.ToLower();
if ($n.EndsWith(".flac") -or $n.EndsWith(".mp3") -or $n.EndsWith(".wma") -or $n.EndsWith(".mp4") -or $n.EndsWith(".wav") -or $n.EndsWith(".m4a"))
{
$flag = 1;
}
}
}
if ($flag -eq 0) { $output += "rmdir " + "`"" + $albumPath + "`" /s /q" + "`n" }
}
Write-Progress "Creating report ... " -status "% Complete" -percentcomplete (100*($i/$list.Length))
}

$output out-file report.txt

Wednesday, October 13, 2010

FLAC Project Finish

FLAC Progress
In a previous post, The Year of the FLAC, we kicked off our home audio CD digitization project. We finished the project on October 1st. First we’ll present some of the parameters of the project, followed by some tactical suggestions if you are considering such a project, and finally some thoughts on the project in general.

Project Parameters

- Number of CDs ripped: 2228. Each CD was read (physically put into a computer disk drive) at least once or more in the case of problem CDs. The 2228 albums were for 1247 artists. The FLACs are stored in the typical folder hierarchy of artists\album.

- Bad CDs: 6 CDs had some bad tracks and 3 CDs were not FLAC’able at all.
- Software: We used dBpoweramp to rip CDs and encode into FLAC.

- OS: Windows 7 on a desktop computer that had two DVD drives which could be operated simultaneously. When using both drives it seemed best to stagger the use of the drives by letting one drive rip while getting the metadata and preparing for the rip on the other drive.

- Size: 643 GB for all the resulting FLAC files (includes imagery which is minor). We calculate roughly a 33% reduction in disk space used compared to the normal audio format (uncompressed PCM raw).

An example of creating a simple PowerShell script for checking for empty music folders after ripping is shown here.

Tactical Suggestions If You Are Thinking About Approaching a Similar Project

- You need a good system of organization when processing this many CDs (or anything for that matter). We used different locations and bins to indicate CDs waiting to be processed and CDs finished. Sticky notes were used freely to leave notes on different CDs because in our effort there were two of us working at any given time.

- You might need to clean a CD in order to get it to read properly. If one drive doesn’t work, try another. We had a couple of cases in which one drive was able to read a CD that another couldn't.

- While Dbpoweramp is pretty good at supplying metadata (it draws from several metadata sources), you should still verify it.

- Spot check rips after completing. There are a couple of cases when a custom post-project PowerShell script that we ran found problems with some tracks. In a nutshell any FLAC file that is less than 500kb is considered suspect as potentially incomplete and we did find some CDs where we thought the CD ripped fine, but didn’t. These infrequent problems are accounted for in the Bad CDs number above.

- If you care about the quality of artwork, the metadata that comes through when using Dbpoweramp may disappoint you, but you can substitute your own. The maximum image size pixel size accepted seems to be about 500 x 500 so anything bigger is scaled to that.

Thoughts on Digitizing a Moderate-Sized Home CD Collection

There is a lot of music we purchased that probably wasn't worth the effort to rip. We will probably never listen to that music and it's perhaps not worth even cluttering, both space-wise and information-wise, our home system. We estimate that about 10-20% of the final music count (2,228) CDs is what we care about. Now with services where you can listen to almost anything you want on demand, we will not being buying CDs in the same way. Yes, we will purchase some, just not as many as we have in the past.

The medium is (part of) the message. A format that you experience music in is important to your connection to it. Part of the angst in the project was letting go of CDs and embracing digital collections like auto-generated playlists (Pandora), music on-demand services (Rhapsody), etc. The Sonos system makes this easy. We get locked into media choices and it becomes hard to change the longer you wait. It seemed inevitable that we took this step. We went through similar pain with a cassette conversion project.

For this project we opened boxes of CDs dated April 2005, when they got packed for our remodel. Wow, not opening a box in 5 years must mean we don’t need or a least we don’t use what’s inside, right? I think the music in those boxes has a greater chance of being used now that it is digitized and easily accessible via Sonos.

Sunday, February 21, 2010

The Year of the FLAC


“She comes out of the sun in a silk dress running like a watercolour in the rain. Don’t bother asking for explanations, she’ll just tell you that she came, in the year of the cat.” Al Stewart.

What does that have to do with this post? Nothing really, other than I think in lyrics too much and this post is about FLAC and I think of 2010 as the year of the FLAC. FLAC stands for Free Lossless Audio Codec. FLAC is a format for storing music digitally that does not lose any of the quality of the music. FLAC is a mathematically lossless format. Most common formats (like MP3s you buy at Amazon) are such that some of the audio quality of the recording is lost. With MP3s we tradeoff a little music quality for smaller file sizes. Most users of digital music don’t mind the tradeoff or won’t be able to tell, especially if playback devices (like computers speakers or an iPod with so-so headphones) are not that great. But, for others the loss of audio quality is annoying and they may want to store music digitally in a format that is the same quality as the audio CD. FLAC can do this.

The FLAC format naturally requires more storage than the common formats (like MP3 or WMA) but still less than what full CD track would. For example, the track by Klaatu Calling Occupants of Interplanetary Craft stored as WMA (Windows Media Audio) at 128kbs (average quality) takes up 6.9 MB. The same track stored in FLAC format is 49.6 MB or about 8 times bigger. But hey, it’s just storage. The real goal is to have all these FLAC tracks available for our friend Sonos.

So far we are about 500+ CDs into a 3,000 CD chore. We are using dBpoweramp to help. Uh, can we just buy these again? At the current rate it will really take a year and it is rightly so the year of the FLAC.

Update 2010-10-12: the conclusion of the project is discussed here.