Lore

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Music Library Organization ======

===== Overview =====

Here is a brief overview of what happens to new content coming in to my library.

  - Tags get cleaned up, any missing metadata is populated, missing album art is retrieved.
  - FLACs are transcoded to MP3s and album art is embedded, then the MP3s are imported to iTunes for syncing with iTunes Match.
  - Processed content is moved to the library once everything looks good.
  - Final confirmation is performed in foobar2000, MusicBee, and iTunes, making sure the number of songs and albums matches.

===== Software Used =====

  * [[https://www.foobar2000.org|foobar2000]] - //foobar2000 is an advanced freeware audio player for the Windows platform.//
    * While it has a loyal following for being a free and highly customizable media player, I find foobar2000 rather cumbersome and dated in terms of usability. Although I've struggled for years to adopt it as my go-to music player, I've grown to appreciate its many utility functions, and rock solid performance.
    * Used to manipulate tags, perform pattern-based renaming and file/folder re-organization, transcode between formats, and to batch embed artwork.
  * [[https://www.foobar2000.org/components/view/foo_discogs|Discogs Tagger]] - //foobar2000 plugin for fetching Discogs.org metadata.//
    * Used to fetch genre information.
  * [[http://lame.sourceforge.net|LAME]] - //LAME is a high quality MPEG Audio Layer III (MP3) encoder licensed under the LGPL.//
    * Used to transcode to MP3. Binaries can be downloaded from [[http://www.rarewares.org/mp3-lame-bundle.php|here]].
  * [[https://www.apple.com/itunes|iTunes]] - //A heaping pile of garbage that unfortunately comes bundled with a single killer feature: [[https://support.apple.com/en-us/HT204146|iTunes Match]]//
    * Used to sync music library to iTunes Match for access on compatible mobile devices.
  * [[https://getmusicbee.com|MusicBee]] - //MusicBee makes it easy to manage, find, and play music files on your computer.//
    * After spending a couple of hours one weekend tinkering with the layout options, I fell in love with MusicBee. It looks great, scrolling through 2000+ albums is fast, it barely uses 100MB of RAM, and has all the playback features I need.
{{ :musicbee_mylayout_1.jpg?direct&400 |My MusicBee layout}}

===== Folder Structure =====

I've adopted this structure at some point and stuck with it. If I had to start over I would probably adjust it a bit, but it works fine. These paths will be referenced throughout the rest of the documentation.

^ Folder                   ^ Description                                                                                           ^
| ''.\_1_new''             | This is essentially the inbox for new content that needs to be processed.                             |
| ''.\_2_cleaned_up''      | Content that is being processed currently will stay here until it is 100% ready to join the library.  |
| ''.\_3_archived''        | This is the library root, foobar2000 and MusicBee monitor it for changes.                             |
| ''.\_3_archived\_FLAC''  | FLAC content gets sorted here.                                                                        |
| ''.\_3_archived\_MP3''   | MP3 content gets sorted here in rare cases when FLAC is unavailable.                                  |
| ''.\_itunes''            | iTunes library lives here.                                                                            |
| ''.\_itunes\_ready''     | Transcodes from foobar2000 go here with intent of importing to iTunes.                                |

**__NOTE:__** All documented paths are relative for brevity and readability, however in practice they must be absolute. For ex: ''Z:\_media\_audio\_1_new''

===== foobar2000 Settings =====

==== MP3 Tag Handling ====

I prefer to avoid ID3v1 and maintain high compatibility for ID3v2, so these are the settings I use.

Navigate to ''Library > Configure > Advanced > Tagging > MP3''

  * ID3v2 revision and quirks
    * ''( )'' Write ID3v2.3 tags (more compatible)
    * ''( )'' Write ID3v2.4 tags (less compatible)
    * ''[ ]'' Use padding (faster tag updates, may be incompatible with buggy software)
    * ''[x]'' Map TPE2 to Album Artist (more compatible)
    * ''[x]'' Write compatible date frames (non-standard-compliant)
    * ''[ ]'' Write rating as TXXX (less compatible)
  * Tag writing scheme for untagged files
    * ''( )'' ID3v2 + ID3v1
    * ''(x)'' ID3v2
    * ''( )'' APE + ID3v1
    * ''( )'' APE

==== Useful Views ====

Navigate to ''Library > Configure > Media Library > Album List''

  * **by album artist/album [by artist]**
    * ''%album artist%|'('%date%')' %album%|%tracknumber%. %title%$if($strcmp(%artist%,%album artist%),,' [by - '%artist%']')''
    * This is a modified default view that detects if //<Artist>// doesn't match //<Album Artist>//, in which case it appends ''[by <Artist>]'' at the end of the relevant tracks. For example //All Cats Are Grey [by - The Cure]// on the //LateNightTales: Air// album.
  * **by album**
    * ''%album% '['%album artist%']'|%tracknumber%. %title%''
    * I believe this is a built-in view. I use it to get a quick album count, and to compare with **by albumsortorder** view.
  * **by albumsortorder**
    * ''%albumsortorder%|%tracknumber%. %title%''
    * //iTunes// keys albums by album name, this causes albums with the same name to get merged in its library. A solution is to use one of the //<*sortorder>// tags, so that //iTunes// uses those as an album grouping key instead. I decided to use the //<albumsortorder>// tag, and this view lets me see the library structured by it. If **by album** and **by albumsortorder** give me the same album count, that means I didn't miss populating //<albumsortorder>// anywhere.
  * **by genre**
    * ''<nowiki>%<genre>%|[%album artist% - ]'('%date%')'  %album%|[[%discnumber%.]%tracknumber%. ][%track artist% - ]%title%</nowiki>''
    * This view is useful for reviewing the genres in the library, and also easily detecting albums with an empty genre tag.

==== File Operations Presets ====

Right-Click on any library item and go to ''File Operations > Rename to''. This will bring up the //File Operations Setup// window with the least clicks. The same window will show for all file operations, just with different pre-populated settings depending on which right-click context menu was selected. Following are the presets I use.

  * **move to cleaned up**
    * Operation type: ''Move''
    * Destination folder: ''.\_2_cleaned_up''
    * File name pattern: ''_%codec%/%album artist%/['('%date%')' ]%album%/%tracknumber%. %title%''
    * ''[x]'' Move entire source folder content
    * ''[x]'' Remove empty source folders
    * ''[ ]'' Show output files when done
    * To save this preset:
      * Under //Presets// enter: ''move to cleaned up''
      * Click: ''Save''

  * **move to archived**
    * Operation type: ''Move''
    * Destination folder: ''.\_3_archived''
    * File name pattern: ''_%codec%/%album artist%/['('%date%')' ]%album%/%tracknumber%. %title%''
    * ''[x]'' Move entire source folder content
    * ''[x]'' Remove empty source folders
    * ''[ ]'' Show output files when done
    * To save this preset:
      * Under //Presets// enter: ''move to archived''
      * Click: ''Save''
    * **Do NOT click** ''Run'' unless you want to perform the operation now.
    * It's safe to ''Close'' the window now.

The output resembles something like this: ''.\_3_archived\_FLAC\Ayreon\(2017) The Source\01. Chronicle 1- The Frame, The Day That The World Breaks Down.flac''

==== Transcode Presets ====

Right-Click on any library item and go to ''Convert > ...''. This will bring up the //Converter Setup// window. Here it's possible to manage a variety of presets, which show up as extra items in the context menu.

Encoder settings are first configured in the ''Current settings'' area, and then it's possible to ''Convert'' right away or ''Save <<'' a preset. If nothing is selected in ''Saved presets'' then a new preset is generated that can be renamed. If a preset is already selected, a popup will appear asking whether to overwrite or create a new preset. Following is the preset I use.

  * **ready for itunes**
    * ''Output format''
      * This area will show a list of presets for various formats. I made my own like so:
        * ''Add new''
        * Encoder: ''MP3 (LAME)''
        * Quality: ''V0''
      * Select the desired format from the list, in my case ''MP3 (LAME) | 245 kbps | V0'', and click ''Back''.
    * ''Destination''
      * Output path: ''Specify folder:'' ''.\_itunes\_ready''
      * If file already exists: ''(x) Overwrite''
      * ''(x) Convert each track to an individual file''
      * Name format: ''%album artist%/['('%date%')' ]%album%/%tracknumber%''
        * I'm only keeping the track number because some of my filenames break 256 character limit. When my //iTunes// performs the import, it will rename the files anyway.
      * Click ''Back''.
    * ''Processing''
      * Not utilized in this preset, but it's possible to perform ReplayGain and other DSP processing here.
    * ''Other''
      * I leave all options disabled except for ''[x] Transfer metadata (tags)''
      * Copy other files to the destination folder: ''*.jpg''
      * Click ''Back''.
    * To save this preset:
      * If overwriting, select preset to overwrite and click ''Save <<'' and then confirm ''Overwrite''.
      * If making a new preset, click in the white-space under the presets, then click ''Save <<'', then rename the new preset. I named mine ''ready for itunes''.

**__NOTE:__** When performing the first transcode with any given encoder, //foobar2000// will prompt for the path of the encoder binary/dll. Check the [[#software]] section to download //LAME//, I chose to keep my binary in ''%appdata%\foobar2000\lame3.100-64\lame.exe''.

===== Step-by-Step =====

==== Initial Cleanup Pass ====

  - Place all new content to be processed in to ''.\_1_new''.
  - Launch //foobar2000// and drag-and-drop ''.\_1_new'' in to an empty playlist view.
  - Select all tracks and ''Right-Click > ReplayGain > Remove ReplayGain information from files''.
    - I just don't utilize ReplayGain, so I rather keep the library clean till I come around to it.
  - With all tracks selected either ''Right-Click > Properties'' or hit ''Alt+Enter''.
    - I don't like unnecessary metadata in my tags, so I delete all tags shown as ''<...>''.
    - I don't keep track of ''Total Tracks'', ''Disc Number'', or ''Total Discs'', so I delete those.
    - ''Performer'' and ''Composer'' are usually not relevant, or don't add any extra information, so I delete them.
    - ''Genre'' will be completely re-done, so I delete it for now.
    - If ''Comment'' has useful information about the release, I'll keep it, but it's usually encoder details, so I delete that as well.
    - Click ''OK'' to apply changes and close //Properties// for now.
  - I need to make sure that ''Album Artist'' is 100% populated, but //foobar2000// shows data in that column from ''Artist'' if ''Album Artist'' is missing, so this requires some manual work.
    - Select all items in the playlist where it's obvious ''Artist'' and ''Album Artist'' would be the same.
    - ''Right-Click > Properties'' or hit ''Alt+Enter''.
    - ''Right-Click'' on //Album Artist// and select ''Automatically Fill Values''.
      - Source pull-down: ''Other...''
      - Source text-field: ''%artist%''
      - Pattern: ''%album artist%''
      - Click ''OK''.
    - Click ''OK''.
    - Manually fix the rest of the items where ''Artist'' and ''Album Artist'' shouldn't match.
  - I prefer dates in ''YYYY'' format, so I eyeball the ''Date'' column in the playlist and correct whatever doesn't conform.
  - I prefer to condense multi-disc releases in to a single album in my library.
    - Some things that I may separate out include:
      - Bonus discs with tracks that break the flow of the original album release.
      - Instrumental versions.
      - Albums where each disk is thematically distinct.
    - For all multi-disc releases that will be merged I need to do the following:
      - Manually continue the track numbering from where Disc 1 ended for all subsequent Discs.
      - Make sure ''Album'' is consistent through the entire release, so delete any mentions of ''(Disc 1)'' and etc...
    - For releases where there will be content separated out, treat the primary album just like above. The separated content will have the same album name, but with an appropriate suffix like ''(bonus disc)'' or ''(Instrumental)''.
  - Before further processing, I need to move the content, because ''Auto track number'' seems to consider the folder structure.
    - Select all content in the playlist, ''Right-Click > File Operations > Move to > move to cleaned up''.
    - The //File Operations Preview// window will come up to show what will happen, eyeball to double check paths and click ''Run''.
    - The content in the playlist will update to reflect its new paths, it's a good idea to sort by ''File path'' and then eyeball the content to make sure everything looks good. If the ''Album'' and ''Track Number'' wasn't adjusted properly earlier for multi-disc releases, they may have merged in to a single folder. Now would be a good time to clean those up and perform another ''move to cleaned up''.

==== Custom Tags ====

  - Select all tracks and either ''Right-Click > Properties'' or hit ''Alt+Enter''.
  - Re-generate the track numbers to make sure they conform to the same style of padding.
    - ''Right-Click'' on //Track Number// and select ''Auto track number''.
    - //Total Tracks// will get re-populated, I don't care for it so I delete it again.
  - For //iTunes// to not merge albums with the same name, but from different artists, in to the same album, I need to add the //<albumsortorder>// tag and populate it with a unique string for each album in the library.
    - ''Right-Click'' in the white-space below the existing tags and select ''Add new field'' or hit ''Ctrl+N''.
      - Field name: ''ALBUMSORTORDER''
      - Click ''OK''.
    - ''Right-Click'' on //<ALBUMSORTORDER>// and select ''Automatically fill values...''
      - Source pull-down: ''Other...''
      - Source text-field: ''%album% - %album artist%''
      - Pattern: ''%albumsortorder%''
      - Click ''OK''.
  - At some point I decided that it may be useful for every track in my library to have a unique //GUID//, one day I'll remember what this was for.
    - ''Right-Click'' in the white-space below the existing tags and select ''Add new field'' or hit ''Ctrl+N''.
      - Field name: ''GUID''
      - Click ''OK''.
    - ''Right-Click'' on //<GUID>// and select ''Edit''
      - Source pull-down: ''Other...''
      - Generate a list of GUIDs and paste it in here. I wrote a PowerShell script to generate these, I'll link to it later.
      - Click ''OK''.
  - Click ''OK''.

==== Album Art ====

  - Check if there is any content remaining in ''.\_1_new'', decide if it should be kept or deleted.
  - Content should all be in ''.\_2_cleaned_up\_FLAC'' now.
  - Search for ''*.*'' and delete any unwanted content, cue sheets, logs, etc...
  - Parse through all album paths manually and confirm presence of album art.
    - The mandatory file is ''cover.jpg'', everything else is extras.
    - If album art is missing, google it.
    - I try to keep the file size and dimensions of ''cover.jpg'' reasonable, ''1000x1000px'' is plenty big.

==== Genres ====

  - The //Discogs Tagger// plugin mentioned in [[#software used]] is required here. I've also mapped ''Ctrl+T'' to the ''Write Tags...'' function of the plugin, to make life easier.
  - These steps have to be performed on individual albums, so select all the album tracks and:
    - ''Ctrl+T'' will bring up the //Find Release// window.
    - Assuming the artist is found, there should be releases listed on the right side. Some artists don't have unique names, and it will require experimenting with the choices in the ''Artists'' area to find the right one.
    - Clicking on an item in ''Releases'' will expand it to show all the available releases //Discogs// has data for.
      - I prefer to pick a release that matches my content's origin, or the closest one.
      - Double-Clicking any release will open its respective page in the default web browser.
      - It's common for me to open pages for a bunch of releases and then review them in the browser to see what is the best match, as the plugin doesn't provide enough info without progressing to the next step.
    - Once a desired release is selected, click ''Next>>''. The //Matching Tracks// window shows side-by-side data from //Discogs// and the selected files.
    - Click ''Preview tags''.
    - If this hasn't been done before, click ''Edit tag mappings...''.
      - Here it's possible to configure which tags get data fetched in to them, and also the various patterns that augment that data. I have all fields disabled, except //Genre// is set to ''write + update''.
      - I use different formatting strings for //Genre// depending on what kind of content I'm processing.
        - Movie soundtracks: ''$unique($append($filter($filter($extend(%<RELEASE_GENRES>%,%<RELEASE_STYLES>%),Score),Stage & Screen),Soundtrack,Movie))''
        - Anime soundtracks: ''$unique($append($filter($filter($extend(%<RELEASE_GENRES>%,%<RELEASE_STYLES>%),Score),Stage & Screen),Soundtrack,Anime))''
        - Game soundtracks: ''$unique($append($filter($filter($extend(%<RELEASE_GENRES>%,%<RELEASE_STYLES>%),Score),Stage & Screen),Soundtrack,Game))''
        - Everything else: ''$unique($filter($filter($extend(%<RELEASE_GENRES>%,%<RELEASE_STYLES>%),Score),Stage & Screen))''
      - In a nutshell, when tagging soundtrack releases, I like to get rid of ''Score'' and ''Stage & Screen'', but I want to append ''Soundtrack'' and ''Movie/Anime/Game''. For all other releases I just get rid of ''Score'' and ''Stage & Screen'' just in case. Also, I merge //Discogs// genre and styles in to a single field, and treat them all as genres. This all is just my preference, and explaining it is a long story.
    - Review the values about to be written in to the tags, and click ''Write tags'' when ready.
      - If any editing is performed to a tag prior to clicking ''Write tags'', the value will be treated as a single item and delimiters will be ignored. This is easy to spot when using the **by genre** view. To fix this, go in to the item properties and manually edit the field, although nothing needs to be changed, just the act of editing will make the delimiters work again. Not sure why this happens, must be some string handling issue. 
    - Quite often //Discogs// has good info in the genres/styles for a release. However, sometimes there is inconsistent information among the various releases for the same master. Due to this I sometimes review multiple releases, and decide whether to manually append/adjust the values I initially fetched with the plugin. Also, I sometimes check //Wikipedia// to further flesh out the genres tag.
    - Sometimes, especially for new releases, the genre/style info is barely fleshed out on //Discogs//, so this is a good time to refer to previous releases already in my library. It's common to just copy/paste data, especially if the artist isn't deviating stylistically with the new release.
  - Repeat the above for every album until finished.

==== Transcode for iTunes ====

  - Select all tracks and ''Right-Click > Convert > ready for itunes''.
    - This will take some time, although LAME is crazy fast and uses every available core/thread.
  - Remove all tracks from the current playlist and drag-and-drop ''.\_itunes\_ready'' in to it.
  - Important to note that I'm now processing the newly transcoded MP3s, not the original FLACs.
  - Select all tracks and ''Right-Click > Tagging > Batch attach pictures''.
    - File lookup pattern: ''cover.jpg''
    - When attached picture already exists: ''(x) Overwrite''
  - Remove all tracks from the current playlist.
  - Import ''.\_itunes\_ready'' in to //iTunes//.
    - After the import, it's safe to delete the contents of ''.\_itunes\_ready'', it should only contain lingering art files.
  - Let //iTunes// sit around for a while, it should automatically start updating //iTunes Match// and //Genius// data.
  - It's a good idea to eyeball the ''Albums'' view in //iTunes// to confirm all the album art shows up.
    - A common mistake I make is to leave ''conver.**png**'' instead of ''cover.**jpg**'' in the folder, and I don't process PNGs when doing ''Batch attach pictures''. In this case I'll convert the PNGs to JPGs for the content I missed in ''.\_2_cleaned_up\_FLAC'', and then manually attach the respective cover art in //iTunes//.

==== Merge and Verify ====

  - Move ''.\_2_cleaned_up\_FLAC'' to ''\_3_archived\_FLAC'', it may be necessary to quit //foobar2000// and //MusicBee// to release any lingering IO handles.
  - Re-launch //foobar2000// and give it some time to detect the new content. I usually keep an eye on the **by album** view, and if the item count at the root doesn't increase after a few refreshes that means it's done updating.
  - The root item count for the **by album** and **by albumsortorder** views in //foobar2000//, and the album count in //iTunes//, should all match.
  - The number of items in the //foobar2000// library should match the number of songs in //iTunes//, excluding any non-music content.
  - Re-launch //MusicBee// and give it some time to detect the new content. Album and song counts should match //foobar2000// as well.
  - I like to create playlists for content added during a certain period, like ''Imported 2017-Fall'', so now is a good time to do that. This is a good way to make sure new content gets appreciated and not immediately lost in a large collection.
  - Rock out.

{{tag>computing music organization}}