User:Billw58/Backup Strategies

From Audacity Development Manual
< User:Billw58
Revision as of 15:49, 27 May 2013 by PeterSampson (talk | contribs) (Archiving finished projects: WIP)
Jump to: navigation, search
Peter 7May12: A use case: this recent forum thread shows the level we maybe should be aiming at: http://forum.audacityteam.org/viewtopic.php?f=16&t=65451

Bill 01May12: @Ed: I prefer the longish introduction section and a short punchy intro div. Also, your alert div makes sense to us but may not make sense to a naive user. Copied it into this ednote and created an advice div in the introduction section.

Alert Always make an external copy of your raw data (original recording), never work with the original material!
  • Gale: 05May12 This looks to me a considerable duplication of existing documentation ( Audacity Projects and Making a backup copy of a project). Wouldn't a FAQ be better? It could point out (as Peter said) that Audacity Projects are not well suited to long term storage (because they are not a unitary format). Hence the emphasis is two-fold: backup a project (if you are even saving one) but for almost all cases, export WAV files. Make clear what extra information the project saves compared to WAV.

    I thought we had agreed, absolutely no dashes?

    • Peter 27May13: I can't see any ruling from Connie on dash usage And ironically the first section of Consistency itself has two uses of the dash as punctuation!  :-)
    • Peter 27May13: I see no real overlap with Audacity Projects and only a small overlap with Making a backup copy of a project which is in the Wiki and not the Manual as this page is intended to be. Both pages could usefully link to this page if and when it gets published. I do not support the idea of this page being a FAQ to my mind it stands as a page in its own right and fully deserves a place in the Manual. The necessarily sectioned nature of this material means that it would not fir well in the FAQs. Nor should it be marooned over in the Wiki imo.
  • Bill05May12: I don't agree that "Audacity Projects are not well suited to long term storage". If they're zipped, what's the problem? As PGA points out in the forum thread, the loss of gain, pan and envelope settings (plus having those settings rendered) is a major drawback of exporting tracks to WAV.
    • Gale: The problem is zipping and unzipping them. A task beyond a significant minority of Windows users.
      • Peter 27May13: I think we've accepted this and dealt with it by inclusion - accordingly I've grayed this bit out.
  • Bill 04May12: Are we all agreed that the advice to "export a WAV of your multi-track project as a backup" is absolutely useless and should be expunged from the Manual (if it still exists anywhere)?
  • Gale 05May12: -1. Of course people should export a WAV for each track of the project.
  • Bill 05May12: The implication, as I recall, was a straight Export (not Export Multiple), which is useless as a backup of a multi-track project. Exporting individual tracks will render the gain, pan and envelope settings which is a major drawback and must be noted.
    • Bill 06May12: Found the statement I was referring to on the Audacity Projects page, the first note div: "If coming back to your project later, it's always a sensible precaution to export a lossless backup WAV or AIFF file of the mix before you quit. "
  • Peter 27May13: We seem to have dealt with this here and on Audacity Projects by recommending saving the project - as we have agreed that Audacity Projects are suitable for long-term storage (provided the user knows what they are doing ... ). Do we want to get into the complexities here of using Export Multiple to export a WAV for each track and the greater complexities of re-threading the project together? I don't.
  • Peter 5May:When we are recommending backup by exporting to WAV below we are implicitly recommending backup to 16-bit PCM WAV. For users working in the default 32-bit float mode in Audacity the best quality capture, and intermediate editing stages, woul be to export as 32-bit WAV surely - but we don't state that.
    Nor can I see any advice about backing up a set of finished production WAVs anywhere in this article. The finished product needs backup protection every bit as much as the capture and intermediate stages - the finished product is the bit that contains the most work after all.
  • Bill 05May12: This what I thought might happen - we all have our own opinions about what the best backup method is. Thus I think we should trim what's here down to a few general, agreed-upon actions.
      • Gale: And put it in a FAQ, taking into account (and if necessary improving) the information we already have to avoid needless duplication. Audacity Projects could use a good cleanup from about halfway down which would leave space to discuss backups.
  • I'd support:
    • The "Introduction" section, emphasizing that Audacity has robust crash recovery.
    • The "Create a folder" section.
    • The "Be Safe" section.
    • Export original unedited recordings to WAV, AIF or FLAC at the same bit depth as Audacity is set for.
      • Why? So you can go back to your original recording in case you make a disastrous edit you cannot recover from.
    • Save incremental backup copies of your project while you are working on it.
      • Why? Same as above.
    • Save a zip archive of the finished project.
      • Why? So you can go back to it at a later date and do further editing. The Audacity project format preserves your gain, pan and envelope settings.
    • And I'm +0.5 on: Export individual tracks as WAV, AIF or FLAC at the same bit depth as Audacity is set for as an archive of your work.
      • Why? In case your zip archive of the Audacity project is corrupted (?)
      • Why not? Gain, pan and envelope settings will be rendered, so this is not a replacement for a zip archive of the Audacity project.
      • I'd then propose we have, on the Wiki, a page where individuals can share their personal opinions and workflows. That page would have a disclaimer that it contains personal user opinions.
  • I made a start on such a page on the talk page. Add your own if so inclined.
This page is about backup strategies particular to Audacity. These should be used in conjunction with, not as a replacement for, regular whole-computer or whole-disk backup procedures.
  • Gale: Regular backups of what? All the computer's data? Or just the recorded file? Or that plus a second backup of the project?
  • Bill 05May12: I meant regular whole-computer or whole-disk backups.
    • Peter 27May13: Tweaked the intro note to fully convey Bill's meaning.
The recommendations on this page are just that: recommendations. Each user should consider their own workflow and develop a backup regime that meets their needs.

Introduction

If you have not already done so, please read the Audacity Projects page, taking particular note of the "Four rules for keeping your Audacity project happy". Backups may not help you if you damage your project by breaking those rules.

Audacity has very robust crash recovery and in most cases will be able to recover your work in case of a power failure, computer crash or crash of Audacity itself. The wise user does not rely on crash recovery, and has appropriate backups in place for those rare circumstances where Audacity's crash recovery can't do the job.

  • Gale: In other words, regularly exported WAV files. Probably a good idea to be explicit here.
  • Bill05May12: That's not what I meant. I was deliberately being vague. Export-multiple-by-track to WAV is not my preference except in the case of a "simple" project. For complex projects incremental backups of the Project structure is my preference.
    • Gale: The majority of users only have one track and never touch gain, pan and envelope. They are the ones most likely to mess up. And I dislike "complex" projects as a term. What is wrong with "multi-track"?

The wise user also knows that hard drives can crash destroying all data and makes regular backups to external storage. The cautious user makes duplicate backups to at least two different external storage devices; the extremely cautious user ensures that at least one of those backups is held off-site.

The nature of audio editing, especially when it involves recording live audio, means that daily backups may not be sufficient. There are also things you can do that are specific to Audacity that will make it easier to make appropriate backups.

Advice Always make a backup copy of a recording (ideally to an external drive) before you start editing your project.
Peter 27May13: ToDo-2 Hasn't the Audio Cache preference been removed from alpha Audacity 2.0.4? I don't see it in the Preferences.
  • If you choose the audio cache option in Directories Preferences to cache data to RAM (memory), no data will be written to disk during a recording. Recoverable data will be written to disk after recording stops, but recordings that crash in progress will be permanently lost.
Peter 27May13: ToDo-2 The Audio Cache preference been removed from alpha Audacity 2.0.4 IIRC & AFAICT thus rendering most of this ednote obsolete. The only part that remains possibly relevant is the textual discussion for the advice note - but I changed that to "Always backup a recording (ideally to an external drive) ... " today
  • Peter 5May12: I don't see the relevance of the audio cache advice to this backup article. The user will not be backing up the project while recording, you can't back up a project while it's recording.
    And should we be saying in the first advice note above "(ideally to an external USB or FireWire hard drive)"
    • Gale: If this is about protecting novices from their own errors, then there will be no recording and nothing to backup if they turn audio cache on and they crash.
    • Peter 6May12: No this is not a tutorial about protecting folk from their errors, nor is it a treatise on the full soup-to-nuts recording process - rather it is an article advising folk how to carefully store the valuable data that thay have created - therefore I really believe that the audio cache stuff has absolutely no place in the artice as it was originally envisaged on the forum thread that PGA kicked off. That doesn't mean that the information is not valid or useful - it just means we need a better home for it elsewher in the manual.
  • Ed 5May12 "ideally to an external USB or FireWire hard drive" Not IMVHO. I use an eSATA external hard drive and an external SCSI tape drive (my FireWire drive is a primary drive not for backup; Lightning drives are available for Mac and soon for us mere mortals); I also use thumb drives, flash memory cards and internal CDrw/DVDrw drives for backup, why be exclusive? My opinion is that the text should be more like:
Advice Always backup a recording (ideally to external media) before you start editing.

Ed 5May12 I do not like two identical style (alert in this case) divs back-to-back; maybe:

    • Gale: I think the Audio Cache most needs the div as it's an aside (but a very important one). So remove the div from the point about backup, or make it one div (I've tried it that way with bullet points but possibly yours below is better.
Advice Always backup a recording (ideally to an external USB hard drive) before you start editing it.


If you choose the audio cache option in Directories Preferences to cache data to RAM (memory), no data will be written to disk during a recording. Recoverable data will be written to disk after recording stops, but recordings that crash in progress will be permanently lost.

Create a folder in which to store your project

Many users prefer to create a new folder on their hard drive in which to store their project; the AUP file and the _data folder are both inside this folder making it easy to keep them together if you need to move them to another location on your hard drive. This also creates a place to store other files associated with the project such as exported WAV backup files and production notes.

Be safe - copy all external files into your project

Before importing a WAV or AIFF file into your project, click on Edit > Preferences, click on the Import / Export section and select selected radio button Make a copy of uncompressed audio files before editing (safer). Later, when you save your project you will not need to think about including those files in the backup.

Backing up single-track projects

These are projects that consist of one stereo or mono track; examples would be digitizing LPs or cassettes, recording an audio book or editing a recorded interview.

Backing up the original recording

Once you have completed the original recording (entire LP or cassette, audio book chapter, interview etc.) click File > Export to export the entire track as WAV, AIFF or FLAC. Save the file in the same folder as the project (not in the _data folder, use the folder that contains the _data folder and AUP file).

Incremental backups during editing

If you are doing extensive editing on the project, periodically export a WAV, AIF or FLAC using a meaningful file name. These exports provide snapshots of your project at the various editing stages. If something goes terribly wrong you can go back to one of these versions instead of starting over.

Archiving the final version

When your project is complete, export another WAV, AIF or FLAC using a name different from the exported file of the original recording. Since your project consists of only one track you can now safely delete the Audacity project. If you ever need to do any further editing you can import this file into a new Audacity project.

One user's experience
A user recorded a one-off live broadcast (that would never be repeated) from a radio station; he applied aggressive noise reduction and compression and exported the result. Sometime later he realized that he had damaged the audio in his zeal to get rid of the noise and by compressing it to make it sound louder. Unfortunately, he had not saved his original recordings so reprocessing was not possible.

Backing up multi-track projects

These projects consist of more than one track; examples of such a project would be a multi-track music production with overdubs, voice mixed with music on another track or an interview where the two parties are on separate tracks.

Backing up the original recordings

At appropriate breaks in the recording session, export each newly-recorded track as a WAV, AIF or FLAC file, giving the files meaningful names.

Incremental backups during editing

Periodically click on File > Save Project As... to save a version of your project with a different but meaningful name; the frequency will depend the complexity of your edits and other factors. The purpose of these multiple projects is to give you a version of your project that you can go back to in the unlikely event that Audacity or your computer crashes while you are in the midst of recording or editing. In most cases Audacity will be able to recover your project in the state it was in just before the crash but there are exceptional cases where Audacity will not be able to do this. Having a previous version that was not being actively worked on at the time of the crash will let you start from that version rather than from the beginning. This is especially important if you are engaged in overdubbing; losing a good performance due to a computer crash or power failure is beyond frustrating. These periodic backups also give you a known state of your project to go back to in the event of a disastrous edit from which you cannot recover.

Archiving the final version

When you are done with a project, create a zip archive of all the files and folders you created while working on the project. If you ever need to go back and make some changes you can unzip the archive, choose the point in the project's life where you want to dive in, and open that project.

One user's experience
A band was doing a multi-track recording session; all went well with the recordings and they had a nice multi-track project with vocals, drums, bass and guitar on separate tracks. During mixdown they applied equalization and compression to the tracks, added reverb to the vocals then saved and closed the project. Sometime later they realized that they had put too much reverb on the vocals. Unfortunately they had not saved a version of the project before they added the reverb. There was no way to remove the reverb from the vocals so their only recourse was to record the vocals again.

Incremental backups

  • Bill 04May12:
    • This seems to be an expansion of the incremental backups alluded to in the above section, but seems to apply to a "simple" project. Backing up each original capture during a multi-track overdubbing session would, IMO, be disruptive to the session. Furthermore, doing an Export-multiple-by-track for a multi-track project may not be appropriate or useful as Audacity no longer preserves the white space at the start of a track so the position of a punch-in is not preserved.
    • I don't get the distinction between "tuneOriginal.WAV" and "tuneRaw.WAV".
      • Ed: just that you want redundancy in the copy of your original raw data
    • The use of the "tune" directory and "tune..." file name is confusing.
      • Ed: That is my way—I would not complain if someone had a better naming convention to offer.
    • At the moment I think we getting too detailed in this section.
      • Ed: Many naive users need this level of hand-holding; I see this as a Tutorial, more than a Tip and the detail is what the user is looking for (IMVHO).
        • Bill: In that case I'd hive this off onto a separate page and reference it from this page, similar to the "Example Workflows" on the Tutorials page. That page could then have input from those of us who have particular backup workflows. This page is not a tutorial page IMO.
  • Gale: You can be pragmatic with a multi-track session. Back up the captures every three tracks when they break for coffee. If it's a band, won't they be able to send a feed to another computer which can record the back up? How far do we go? Uninterruptible power supplies?

    Did Audacity ever preserve white space at the start when exporting? 1.3.11 (well before Richard's changes) doesn't. What is different is that Mix and Render is no longer a solution because it does not render the initial white space as silence, which would preserve the lead-in. So we just tell them to double-click close to the start of the audio, LEFT, hold SHIFT then press HOME, then Generate > Silence.

    I would recommend naming the incremental projects and WAV's with the date and time.

  • Bill 05May12: This is the problem - how specific do we get. I started this page so we'd have a basis for discussion; I'm not proposing that everything on this page needs to be here. In fact, it may turn out that this should be on the Wiki as "recommended workflows" since everyone has their own ideas of what is appropriate.
    Yes, for multi-track projects, at break time, doing an export-by-track of recordings since the last break is doable. But, again, how specific do we want to get?
    • Gale: I really do not think a page in "Using Audacity" should be devoted to teaching people common sense or about the need to treat all digital media as fragile.
  • Bill 06May12: I'd lose the "Incremental Backups" section. Too detailed.
    • Peter 27May23: ToDo-2 I'm minded to partly agree with Bill here - but I'd retain the basic text and lose the "examples" starting from "; as a result you might have something like: ... "

Once you have gotten to the point of having some audio recorded (or imported), before doing anything else, you should save both a WAV (a lossless uncompressed, FLAC format is another possibility) file of your raw recording (for details see Export) and then an Audacity Project (for details see Saving).

Make your edits; if you are making a lot of changes save a new Project with a different name--use menu File > Save Project As... from time to time (this will give you incremental backups on disk). At least once a day (more often if desired) save the Project on an external drive.

When you are happy with all your edits, make a final Save Project As... under a new name.

Peter 27May13: commenting this out
as a result you might have something like

C:\tune\tuneRaw.aup
C:\tune\tune1.aup
C:\tune\tune2.aup
C:\tune\tune3.aup
[...]
C:\tune\tuneFinal.aup
C:\tune\tuneOriginal.wav
C:\tune\tuneRaw.wav
C:\tune\Tune.wav
C:\tune\Tune.mp3

on your hard drive and:

T:\tune\tune4.aup
T:\tune\tune9.aup
T:\tune\tune12.aup
[...]
T:\tune\tuneFinal.aup
T:\tune\tuneOriginal.wav
T:\tune\tuneRaw.wav
T:\tune\Tune.wav
T:\tune\Tune.mp3

on an external drive and also on one (or a series of) CD(s) or DVD(s).

C:\tune\tuneOriginal.wav is the original raw recording
T:\tune\tuneOriginal.wav is a copy of the original raw recording
C:\tune\tuneRaw.wav is the working copy of the original raw recording
C:\tune\tune1.aup is the original Audacity Project with no edits—just the original audio
C:\tune\tune2.aup is the first incremental backup Audacity Project with edits
T:\tune\tune4.aup is the first incremental backup Audacity Project stored externally

note: we do not store every incremental backup externally

C:\tune\Tune.wav is the final production version of the audio ready to be burned to CD
C:\tune\Tune.mp3 is the final compressed version of the audio ready to be e-mailed or put on an iPod®

note: both Tune.WAV and Tune.mp3 are stored externally

Archiving finished projects

Bill 06May12: I think we can lose this section, since it is now covered in the "Multi-track projects" section.
    • Peter 27May13: Originally I thought I agreed with Bill here - but I changed my mind and think we should keep it. Plus I will add some additional material on 32-bit WAV export.

One safe and sure way of keeping your project files and folders together is to make a zip archive. Save the zip file on an external drive, CD-R or DVD-R if it will fit, or, better yet, both. If you ever need to revisit the project you can unzip it and continue editing, overdubbing or whatever.

If you are working in Audacity's default 32-bit float mode, the best quality capture and best for editing, than you may wish to retain that quality for the raw capture and the finished production file in 32-bit WAV format.