[Coco] RBF File System Questions

[Gene Heskett]

On Tuesday 29 May 2007, Darren A. wrote:
>>From: Gene Heskett
>>Subject: Re: [Coco] RBF File System Questions
>>Date: Mon, 28 May 2007 23:53:42 -0400
>>
>>  <snip>
>>
>>One way to help the fragmentation problem that I'm fond of, is to reset
>>that
>>default PD.SAS from 8, to as high as FF(hex), forcing the filesystem to
>>locate and allocate as many as 255 sectors in one swell foop.  The file may
>>not actually use that many, and the leftovers will be returned to the free
>>space pool when the file is finished and closed.  This is far faster for
>>large file operations than having it open the default, claim 8 clusters,
>>write them, find its out of disk space and then have to do that again 33
>>more
>>times.  And yes, that is in clusters, which may not be a 1, and could be
>>any
>>power of 2 that will fit in a one byte definition.  Of course, when the
>>disk
>>is nearly full, then that particular scheme of using a large value for
>>FD.SAS
>>has to be scaled down.  One can use the free command to determine the ideal
>>size in that event.
>>
>>In that regard, dmode is your friend.
>
>--
>
>I found an old (1990) document on the web that seems to say that the SAS
>value is not in clusters. Can anyone confirm which case is true?  Here is
>the relevant excerpt:
>
>----
>A "cluster" is a lot like the minimum sector allocation size except that it
>is the same for all types of files and it can't be changed on the fly.  If
>the cluster size is set to 32, every file will allocate a multiple of 32
>sectors (actually 31 sectors p lus 1 file-descriptor sector for the first
>cluster) and remain there until it needs another cluster.

This statement below doesn't grok.

>As far as how cluster size and sector allocation size (SAS) work together,
>the initial file size is always 1 cluster.  On disks with 1-sector clusters,
>this is used exclusively by the file-descriptor.  After that, as soon as
>data is written past the end of a segment, more space is allocated to the
>file in chunks of SAS rounded up to the next cluster size.  (eg. if cluster
>size=$08 and SAS=$23, additional space is allocated in chunks of $28
>sectors).  When the file is closed, it is shrunk to the least used number of
>clusters.

I am the one who converted the rbf.mn to both operate on a 6309, and restored  
the functionality vis-a-vis proper handling of multiple sector sized 
clusters, and as such I had to dis that puppy, looking up obscure defines etc 
so that the proper defines rather than some arbitrary offsets were used when 
it was re-assembled.  Kevins famous christmas patch removed about 80% of that 
functionality, but by dissing the original version of rbf.mn that came with 
the first level one distribution, I was able to find the correct code and 
restore it to the versions I did.

Unforch, I also made a modification that was a time-bomb that aided the 
housekeeping for the 'BackAR' program, and which at the end of the day 
destroyed the file system structures of several disks for folks who did not 
understand the caveats about the default FD.SAS=8.  Getting rid of that was a 
3 byte nop, nop, nop patch once the mechanism was understood.

First, divorce the word 'cluster' from the word 'sector' in the thinking.

Cluster size in the default descriptor, and as written to the disk by the 
normal format comand is $01, meaning that the two words are at that point 
interchangeable.

In fact, I'm not sure how one could go about giving the format command the 
option to format a disk with a cluster size other than 1 as all of my 
multiple segment clustering tests I did at the time were performed on a 720 
dsdd disk made into a cluster size power of 2 with ded /d1@ making the 
changes required to make a multiple sector cluster system out of it after the 
formatting process was completed.  If indeed 'format' is capable of doing 
that by itself, then a tutorial needs to be written about that.

I did not have the luxury of a second hard drive big enough to allow any 
experimentation, and at the time my drive was a maxtor 7120s, which formatted 
to about 129 megabytes, so the FAT on that disk actually did use some of the 
last sector of a 65536 cluster setup that rbf can handle.

So the initial disk area allocated when a file is opened for writing will be 
FD.SAS * cluster size, so if FD.SAS=$20, then the initial disk allocated will 
be 32 * (1,2,4,8,16,32 yadda, yadda), so it could be 32, or 64, or 128, or 
256, 512, 1024 sectors depending on the cluster size of that particular disk.
The File descriptor sector will be the first sector used, the file will occupy 
the rest, and if the file needs yet more disk, than another FD.SAS * cluster 
size block of sectors will be allocated, and another 5 byte FD.SEG will be 
written to the file descriptor.

This can continue until you are out of FD.SEG's with the 48th and final 5 byte 
entry in the file descriptors sector.  Or out of disk, which is indicated by 
a failure to find, in the FAT, sufficient, contiguous free clusters to 
satisfy another FD.SAS allocation.  During the formatting, those bytes in the 
last sector of FD.FAT that are beyond the end of the disk are set to $FF.  
For single sector cluster 720k floppies, then FD.FAT is 2, and much of the 
second sector is set to $FF.

When the file is closed, any unused CLUSTERS are returned to the FAT, and the 
last FD.SEG entry is re-written to reflect its less than FD.SAS clusters 
usage of the disk, thereby maintaining the integrity of the file system.

So basicly, the disk size is scaled to fit into a 65536 byte FAT by the 
cluster size, and all disk housekeeping is based on the cluster size.  All 
other file operations are still based on direct sector addressing.  But 
because you need to have more than 134,217,728 bytes of disk to manage before 
you need to change to a cluster > 1, it has not until more recently, been 
further explored.

I do not believe there were any logic errors remaining in my final, but 
infamously disastrous because of the BackAR 'bug', version of rbf.mn.  I 
tested it very thoroughly at cluster sizes up to and including 8, with many 
different FD.SAS's in effect, so each test took many hours to perform, with 
me looking at the disk with ded by hand, several times during the testing at 
each cluster size to ascertain that it was indeed being written correctly.

Hopefully this will clarify some of the fog that seems to surround the sectors 
vs clusters thinking.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
It isn't easy being the parent of a six-year-old.  However, it's a pretty 
small
price to pay for having somebody around the house who understands computers.