Zoom H5 | Repairing RIFF WAV files recovered from an SD card that I accidentally formatted

Discuss the Zoom H6, H5, H4, H4n, H2, H2n, and H1. Please don't "post and run". Participate in the discussion. Thanks.
Post Reply
ngraham
new to this board
new to this board
Posts: 7
Joined: Tue Aug 15, 2017 10:51 am

Zoom H5 | Repairing RIFF WAV files recovered from an SD card that I accidentally formatted

Post by ngraham » Wed Aug 16, 2017 11:58 pm

Recently I transferred several recordings from a Zoom H5 to my computer hard drive, formatted the SD card that the original files were recorded to, and then accidentally deleted the files off my computer permanently (I know—rookie mistake / big oops). These recordings were the vows and speeches for my good friend’s wedding (intended to accompany some video footage we shot), so I'm really hopeful that I can get them back. I tried to recover the files from my hard drive using data recovery software, but was unsuccessful. So I tried to recover from the formatted memory card, and had a little bit more success. I was hoping to find a series of *.WAV files, but instead found a number of *.RIFF files. I don’t know what process the H5 uses to ultimately create the WAV files, but perhaps it creates files with a *.RIFF extension as an interim step?

Anyway, some of the files are actually completely useable, just by changing the file extension from *.RIFF to *.WAV, but in other files, I can hear audio, but it’s interrupted by blips of interference at what seems to be predictable intervals (fractions of seconds)—it sounds like a CD skipping. From what I can tell, this appears to be happening on files where I’ve recorded multiple tracks (e.g. L/R from the built-in Mic, and Line 1 in from a mixer). The files that are fine are the ones with only the L/R channel recording. I’m not sure what’s happening, but I’m wondering if the H5 records all channels to a single RIFF file before splitting them into multiple WAV files.

Does anybody have any ideas on how to repair these WAV files? Any help is greatly appreciated!
0 x

Wulfraed
Yoda
Yoda
Posts: 2946
Joined: Wed Jan 23, 2008 8:18 pm
Location: Kentwood, MI

Re: Zoom H5 | Repairing RIFF WAV files recovered from an SD card that I accidentally formatted

Post by Wulfraed » Thu Aug 17, 2017 7:46 am

ngraham wrote:
Wed Aug 16, 2017 11:58 pm
card, and had a little bit more success. I was hoping to find a series of *.WAV files, but instead found a number of *.RIFF files. I don’t know what process the H5 uses to ultimately create the WAV files, but perhaps it creates files with a *.RIFF extension as an interim step?
The RIFF extension was likely created by your recovery software. RIFF is an "Interchange File Format" (IFF files were the "native" type used on the Amiga for music, audio, images, etc. -- but used the Motorola byte order for large integers; Microsoft [as I recall] created RIFF to be similar but using Intel byte order).

Your recovery software read the first part of the file and didn't go any deeper (IFF files can contain multiple types of embedded data); for example, here is the header for a WAV file I had lying around:

RIFFø¿šWAVEbextZ Me and Bobby McGee

The garbage characters are length fields
Anyway, some of the files are actually completely useable, just by changing the file extension from *.RIFF to *.WAV, but in other files, I can hear audio, but it’s interrupted by blips of interference at what seems to be predictable intervals (fractions of seconds)—it sounds like a CD skipping. From what I can tell, this appears to be happening on files where I’ve recorded multiple tracks (e.g. L/R from the built-in Mic, and Line 1 in from a mixer). The files that are fine are the ones with only the L/R channel recording. I’m not sure what’s happening, but I’m wondering if the H5 records all channels to a single RIFF file before splitting them into multiple WAV files.
No -- it writes two WAV files... Problem is that the data gets flushed to the card at fairly regular intervals for each file, so the allocated data blocks end up alternating in the card. The recovery software is seeing the start of "a" file, and is taking all the following data as part of that file until it runs into something that indicates the end (if it uses the RIFF length, the result will be about half of the real files -- and no idea how the other half will be handled). I'd suspect that, looked at with a binary editor, the second file's header will appear one block into combined file.

Does anybody have any ideas on how to repair these WAV files? Any help is greatly appreciated!
Essentially, you have to determine the correct block length, and write all the odd-number blocks to one file, and the even number blocks to another file.
0 x
--
Baron Wulfraed
IISS Elusive Unicorn (detached)

Superscope PSD-300; BOSS BR-600, Zoom HD16cd, Zoom R16, BOSS BR-800, Zoom H2n
Now to (re)learn to play an instrument

Lanikai S-C, SMC-E; GoldTone Banjo-Uke; Flatiron 1C, A5; Big Muddy M1-W; Ovation MM68AX, CSE-44; Orpheus Valley Fiesta FS; Taylor NS-72ce, T5-S1; Musima (4st, 20 fret, tenor-tuned) banjo; bongos, dumbeks, bodhrans, hand drum, tambourine; recorder: soprano, alto, tenor; Cedar Flute (5 sizes); Pennywhistle (3 keys); Casio keyboards

ngraham
new to this board
new to this board
Posts: 7
Joined: Tue Aug 15, 2017 10:51 am

Re: Zoom H5 | Repairing RIFF WAV files recovered from an SD card that I accidentally formatted

Post by ngraham » Thu Aug 17, 2017 10:17 pm

The RIFF extension was likely created by your recovery software. RIFF is an "Interchange File Format" (IFF files were the "native" type used on the Amiga for music, audio, images, etc. -- but used the Motorola byte order for large integers; Microsoft [as I recall] created RIFF to be similar but using Intel byte order).
Very helpful—thanks!
I'd suspect that, looked at with a binary editor, the second file's header will appear one block into combined file.
Looking at the file in a hex editor, this is definitely what's happening. There's a header that starts at the first byte, and then a second header that starts at byte 32768. The actual data starts at 65536.
Essentially, you have to determine the correct block length, and write all the odd-number blocks to one file, and the even number blocks to another file.
Hoping these blocks are indeed at regular intervals. Now I just need to figure out how to split the file into two. Fingers crossed. Will report back, but thanks for your help in the meantime!
0 x

Wulfraed
Yoda
Yoda
Posts: 2946
Joined: Wed Jan 23, 2008 8:18 pm
Location: Kentwood, MI

Re: Zoom H5 | Repairing RIFF WAV files recovered from an SD card that I accidentally formatted

Post by Wulfraed » Fri Aug 18, 2017 8:24 am

I suspect the actual data does start a bit sooner -- the header itself is not 32kB long; what you saw at 64kB is the start of the second block of the first file.

https://en.wikipedia.org/wiki/Resource_ ... ile_Format
https://en.wikipedia.org/wiki/WAV#RIFF_WAVE

(and just for historical purposes: https://en.wikipedia.org/wiki/Interchange_File_Format )

Given a known (and unvarying) block size, the algorithm basically becomes something like (pseudo-code based on Python):

Code: Select all

BLOCKSIZE = 32768	#whatever the actual block size is

fin = open("corrupt.file", "rb")	#needs to be a binary, not text I/O system
fout1 = open("A-part.file", "wb")
fout2 = open("B-part.file", "wb")

current = fout1	#I'm going to get tricky below

while True:
	chunk = fin.read(BLOCKSIZE)
	
	if not chunk:	#end of file
		break

	current.write(chunk)

	if current is fout1:
		current = fout2
	else:
		current = fout1
		
fout1.close()
fout2.close()
fin.close()	
0 x
--
Baron Wulfraed
IISS Elusive Unicorn (detached)

Superscope PSD-300; BOSS BR-600, Zoom HD16cd, Zoom R16, BOSS BR-800, Zoom H2n
Now to (re)learn to play an instrument

Lanikai S-C, SMC-E; GoldTone Banjo-Uke; Flatiron 1C, A5; Big Muddy M1-W; Ovation MM68AX, CSE-44; Orpheus Valley Fiesta FS; Taylor NS-72ce, T5-S1; Musima (4st, 20 fret, tenor-tuned) banjo; bongos, dumbeks, bodhrans, hand drum, tambourine; recorder: soprano, alto, tenor; Cedar Flute (5 sizes); Pennywhistle (3 keys); Casio keyboards

ngraham
new to this board
new to this board
Posts: 7
Joined: Tue Aug 15, 2017 10:51 am

Re: Zoom H5 | Repairing RIFF WAV files recovered from an SD card that I accidentally formatted

Post by ngraham » Fri Aug 18, 2017 1:53 pm

Thanks for the help! I’ll see if I can figure out how to implement and revert!
0 x

ngraham
new to this board
new to this board
Posts: 7
Joined: Tue Aug 15, 2017 10:51 am

Re: Zoom H5 | Repairing RIFF WAV files recovered from an SD card that I accidentally formatted

Post by ngraham » Mon Aug 21, 2017 2:11 pm

Okay, I successfully implemented the code and split the file into two parts, alternating every 32768 bytes. However, both of the output files still have the same issue (audio is being interrupted at seemingly regular intervals by other audio). Really curious what's going on here. A few other observations on the files / process below:
  • To reiterate, it's only the multitrack recordings that are problematic.
  • For the multitrack recordings in question, the original recordings captured both the XY stereo mic and the line 1 input (i.e. one stereo track and one mono track), so two files would have been created on the card for each recording.
  • Looking at one particular file as an example, there were two files recovered that relate to the same recording; let's call them A.RIFF and B.RIFF.
  • A.RIFF is 526MB, B.RIFF is 1.05 GB (2x size of A.RIFF).
  • Looking at uncorrupted files recorded in the same fashion, I'd expect to see two files, with each file containing a unique header followed by data.
    The 2x relationship appears to exist for the uncorrupted files as well (i.e. stereo file size = 2x mono file size).
  • In the corrrupted files, A.RIFF and B.RIFF, there are three headers between the two files. Two in FILEB.RIFF, and one in A.RIFF. It looks like the second header in B.RIFF (starting at byte 32768) is the same header as in A.RIFF. In fact, it looks like the data that follows the second header in B.RIFF is actually the same as the data that follows the header in A.RIFF, just offset by 32768. To confirm this, I picked a few lines at the end of A.RIFF, and went to the same lines in B.RIFF (offset by 32768 bytes), and they were the same.
... And now my head hurts. Any ideas what's going on here?
0 x

Wulfraed
Yoda
Yoda
Posts: 2946
Joined: Wed Jan 23, 2008 8:18 pm
Location: Kentwood, MI

Re: Zoom H5 | Repairing RIFF WAV files recovered from an SD card that I accidentally formatted

Post by Wulfraed » Tue Aug 22, 2017 8:18 am

Ouch...

The only thing left in my mind is that the built-ins created one stereo WAV file, while the external inputs are generating a pair of mono WAV files -- ie; a total of three files.

If so, this could get trickier:
  • Are the segments always in the same order (two alternating files is easy, but three that are not ABCABCABC would require hand assembly
  • Are the mono files written in the same block size (since stereo takes twice the data, one could expect two blocks of stereo file for each pair of mono file blocks (So: AaBCAaBC, or ABaCABaC, or other combination)
Possibilities are many... Maybe try modifying the algorithm to rotate through three output files... And then (if stereo is two blocks of data per one block each external) trying to determine which blocks in the sequence are for the stereo file and having the algorithm do the appropriate cycling (A B A C perhaps)
0 x
--
Baron Wulfraed
IISS Elusive Unicorn (detached)

Superscope PSD-300; BOSS BR-600, Zoom HD16cd, Zoom R16, BOSS BR-800, Zoom H2n
Now to (re)learn to play an instrument

Lanikai S-C, SMC-E; GoldTone Banjo-Uke; Flatiron 1C, A5; Big Muddy M1-W; Ovation MM68AX, CSE-44; Orpheus Valley Fiesta FS; Taylor NS-72ce, T5-S1; Musima (4st, 20 fret, tenor-tuned) banjo; bongos, dumbeks, bodhrans, hand drum, tambourine; recorder: soprano, alto, tenor; Cedar Flute (5 sizes); Pennywhistle (3 keys); Casio keyboards

ngraham
new to this board
new to this board
Posts: 7
Joined: Tue Aug 15, 2017 10:51 am

Re: Zoom H5 | Repairing RIFF WAV files recovered from an SD card that I accidentally formatted

Post by ngraham » Tue Aug 22, 2017 5:06 pm

See, I don’t think there were actually three files written to the card. As I mentioned before, I was recording only one line input and one stereo mic input. When I make other recordings in this fashion, it only creates two files (unless the file size exceeds 2 GB, which is not the case here). It seems like the smaller file is actually fully contained by the larger file (i.e. the 526 MB of data following byte 32768 in the larger file are the same as the 526 MB of data that comprise the smaller file). Is there any explanation that would support this? Perhaps something to do with the way the files were recovered? Or maybe related to what happens when the Zoom performs the “Format Card” operation?
0 x

Wulfraed
Yoda
Yoda
Posts: 2946
Joined: Wed Jan 23, 2008 8:18 pm
Location: Kentwood, MI

Re: Zoom H5 | Repairing RIFF WAV files recovered from an SD card that I accidentally formatted

Post by Wulfraed » Wed Aug 23, 2017 9:10 am

If you are actually seeing three "headers", then at some point in time there had to be three physical files, one for each "header".

I could understand a case where the recovery software found two /directory/ entries, where the second entry pointed at a second "header" and "recovers" two files. One file containing both "headers", and the second file only having the second "header" but otherwise duplicating everything else...

But for three headers in /one/ file? Recovery software shouldn't be duplicating blocks already visited, so the three headers were in the memory card. The only other way to get three headers -- but they should be quite far apart -- is if the original I/O was triggering a card reallocation (Flash memory erases in large allocation blocks, but can write in smaller sectors -- but writes are one-time, to rewrite a sector, the card finds/erases a free allocation block, copies everything from the old block except it supplies the new sector data as needed, then puts the old block on the free list).

High end cards are capable of keeping multiple allocation blocks "open" (so multiple files could be written to different allocation blocks without triggering reallocations); cheaper cards only handle two allocation blocks (and one will tend to be used for the FAT/Directory data, so only one is available for data I/O). Card CLASS is not a criteria -- Class 10 cards are tested for streaming video recording -- which means only one active data file, and continuous I/O; 2 open allocation blocks is all that is needed; Class 2/4/6 cards were rated using small/multiple files -- still image cameras with deleted files leaving "gaps" -- so a high-end Class 6 with 4+ open allocations could be faster than a low-end Class 10 when using multiple files.

If the card you had does the erase pass when "opening" an allocation block, not when putting a block on the free list, the recovery software might be seeing the real block and the freed block as both containing valid data for a file.

And since you had formatted the card -- you wiped out the FAT/Directory, so any recovery software is probably working on the basis that any block that contains zeroes (an erased Flash memory is set to all one bits, writing converts a one to a zero, but can not convert in the other direction, hence the need to re-erase to rewrite) is unused. This is a bit of a difference from delete, which typically just sets a flag in the FAT/Directory to indicate sectors are available for reuse and the directory slot is empty.
1 x
--
Baron Wulfraed
IISS Elusive Unicorn (detached)

Superscope PSD-300; BOSS BR-600, Zoom HD16cd, Zoom R16, BOSS BR-800, Zoom H2n
Now to (re)learn to play an instrument

Lanikai S-C, SMC-E; GoldTone Banjo-Uke; Flatiron 1C, A5; Big Muddy M1-W; Ovation MM68AX, CSE-44; Orpheus Valley Fiesta FS; Taylor NS-72ce, T5-S1; Musima (4st, 20 fret, tenor-tuned) banjo; bongos, dumbeks, bodhrans, hand drum, tambourine; recorder: soprano, alto, tenor; Cedar Flute (5 sizes); Pennywhistle (3 keys); Casio keyboards

Post Reply

Who is online

Users browsing this forum: Baidu [Spider], Bing [Bot] and 1 guest