How to read MCCD raw data tapes


========================================================================

The original MCCD data format is raw tape I/O of the individual scans
onto Exabyte tape.  The actual data file format is FITS with one
nonstandard feature - the exact time of each CCD image is stuffed into the
first few bytes of the image.  This is the cause of some of the "bad"
pixels at the beginning of each cube.  In addition, the orientation of the
cube is not intuitive for interactive display, as the CCD orientation in 
the spectrograph is 90 degrees rotated.  Therefore one of the functions
of the tape reader is to swap the CCD axes.

The general idea is:
	Read the tape onto a disk
	Rename all the files to current convention
	Make quicklook movies
	Write the data out to TAR format tape or copy to DvD.

There are two tape sources: CAM, the spectrograph CCD camera
			    FG,the slit monitor video camera frame grabber

The old Exabyte tapes only hold 2GB, while the new Mammoth
holds about 28GB.  In fact, given the hours covered by a single observer,
the CAM tapes only have about 1.1GB at most, and the FG tapes have
about 300MB at most.

The biggest single problem may be that some tapes do not read reliably.
Usually if there is a problem it is the first data file on the tape
that has a problem, and you can skip over it to get the rest of
the images. (Yes, that is the long, good seeing first file.  We are
working on the new datasystem that will eliminate the old tape drives.)
Sometimes the tape will read on a different drive.  It also happens that
some bad patches may take a long time to read as the drive retrys with
different settings, but that is OK, just let it run as long as it can.
I have never been able to skip the bad file and work backward
using backspace record - maybe someone will figure out how to do it.


I. Read in the data files  ====================================

1) Get the tape reader

	/solar/Mees/MCCD/bin/mccd_rd_tape

	in your path (saves typing)  and get SSWIDL setup

	You can do this once in your .cshrc file and .idl_startup file.


2) Set up the disk directory

	The disk is represented as DISKNAME here; it might be /scrd,
	for example.

	a) cd DISKNAME
	b) mkdir CCYY (CC the century, 19 or 20, YY the year), e.g. 2001
	c) cd CCYY
	d) mkdir CCYYMMDD  (mm the month, 01 to 12, dd the day, 01 to 31)
	e) cd CCYYMMDD
	f) mkdir CAM
	g) mkdir FG  (if you have this tape and plan to read it)


3) Read the CAM tape

	This set of instructions with use DRIVENAME to represent
	the name of the tapedrive that does raw I/O, without rewinding.
	On a Sun system, it might have a name such as /dev/rmt/1n;
	this would commonly be called drive 1, which is represented
	here as DRIVENUMBER.

	The name of the MCCD tape reading program is represented as
	mccd_rd_tape.  This may differ depending on the vintage
	of the code.

        a) cd CAM 

        b) Load the CAM tape into the Exabyte drive.

        c) mt DRIVENAME fsf 1
		(Tapes start with a tape mark, usually
                 called an end-of file or EOF, that
                 must be skipped.)

        d) mccd_rd_tape DRIVENUMBER

                file file1 0
			(This program is old and dumb.  If you
			do not set the filename with the file command,
			data from every tape file will be written
			with the same names, overwriting earlier
			data.)

                get 1000 0

        e) Repeat step (d) for as many files as are listed on the printout
        sheet that comes with the tape.
		(This program crashes because it cannot
                 tell the difference between an EOF and
                 an error.  That is why it must be rerun for each
		 tape file.)

        f) mt -f DRIVENAME rew
		(Rewind the tape)

        g) Unload the tape - just push the button and wait.


4) If desired, read the Frame Grabber tape

	a) cd FG
	
	b) Load the FG tape into the Exabyte drive

	c) mt -f DRIVENAME fsf 1 

	d) mccd_rd_tape 1

		file file1 0

		get 1000 0

	e) Repeat step (d) for as many files as are listed on the printout
	sheet that comes with the tape.

	f) mt -f /dev/rmt/1n rew  (rewind the tape)
	
	g) Unload the tape



II. Rename the files  ====================================

	a) sswidl
		(Starts IDL with the SolarSoft paths and programs.)

	b) files = FINDFILE('DISKNAME/CCYY/CCYYMMDD/CAM/fi*')
		(Gets the names of all the disk files.)
	
	c) help, files   (Make sure there are some - check against typos)

	d) mccd_rename, files
		(Reads the FITS headers, generates correct names, renames
		 the files.)

	e) files = FINDFILE('DISKNAME/CCYY/CCYYMMDD/FG/fi*')  (If desired.)

	f) help, files
	
	g) mccd_rename, files



III. Make the quicklook movie  ====================================

	This movie is put into a directory you specify, represented
	as MOVIEDIR here. It
	is easy if this is a directory on a Web site, so everyone
	can view the movie with their browser.

	a) cd, 'CCYY/CCYYMMDD/CAM'     (move into the directory)

        b) mccd_movie_start, moviedir='MOVIEDIR'
		(This tries to make a quicklook movie by taking
		a sample of datacubes through the day.  If the
		MCCD pointing is changed during the day, that
		will be seen.  If the active region is changed,
		the movie will only be made on the region that
		was observed the longest.  To make the movie
		for the other region, move the files for that
		region into a temporary directory and run the movie
		maker there. Be sure to put the files back.)

		To view te movie, point a Web browser at the
		mccd_ARnnnn_CCYYMMDD .html
		file that it produces in MOVIEDIR.
		

        c) exit (Get out of IDL)


IV) Copy the data

	To copy the data onto a TAR format tap, use the 
	MCCDarchive program.

	To copy the data onto DvD, TBD.

======================================================================

Barry LaBonte
     Last modified: Wed Jun 25 12:13:35 HST 2003