Full Version: Limit to amount of folders a RaQ can handle?
Is there a limit to how many folders a RaQ can handle in the main directory (like http://www.whatever.com/name)? Could it handle say 500,000 directories?
Well, yes and no.
Recently I wrote a script that created a null file (touch blah/blah) every time an event happened (don't ask why I did it like this, I wasn't thinking).
Well, on one particularly hard day, it created around 70,000 files in the directory.
Well every night I had a cron job running that deleted those files and started a new day.
Well, I'm not sure how many is "too many" but when you rm * in a directory with too many files, it says "file list too long" and rm exits without deleting anything.
I didn't know that was happening until my harddrive filled up (9 gig drive) at 71% (yes 71% full but it was full).
It wasn't "counting" the file system entries for the "null" files so I had (roughly) 420,000 files in that directory and that ate up the last 3 gigs or so of my harddrive.
It took me hours to delete them all (remember give rm more than a "few" (20? 30?) thousand files to delete and it bugs out).
So. Yes you probably "could" put 500,000 directory entries, you'd probably use several gigs of harddrive in directory "FAT" entries alone, and it'd take you patience and HOURS to delete them (assuming you ever wanted to delete them).
Not sure what Apache would think about all this however. So the answer could very well be "nope".
Recently I wrote a script that created a null file (touch blah/blah) every time an event happened (don't ask why I did it like this, I wasn't thinking).
Well, on one particularly hard day, it created around 70,000 files in the directory.
Well every night I had a cron job running that deleted those files and started a new day.
Well, I'm not sure how many is "too many" but when you rm * in a directory with too many files, it says "file list too long" and rm exits without deleting anything.
I didn't know that was happening until my harddrive filled up (9 gig drive) at 71% (yes 71% full but it was full).
It wasn't "counting" the file system entries for the "null" files so I had (roughly) 420,000 files in that directory and that ate up the last 3 gigs or so of my harddrive.
It took me hours to delete them all (remember give rm more than a "few" (20? 30?) thousand files to delete and it bugs out).
So. Yes you probably "could" put 500,000 directory entries, you'd probably use several gigs of harddrive in directory "FAT" entries alone, and it'd take you patience and HOURS to delete them (assuming you ever wanted to delete them).
Not sure what Apache would think about all this however. So the answer could very well be "nope".
QUOTE
500,000 directory entries
Man you have alot of pages, huh? ;-)
First: on Linux every directory is like a file in general, it means it needs storage for the name and for general data, let's say if that would be 32 Bytes, you would have 16 Mbytes just of useless folder data. I'm not aware of how much folders the system can handle, but you can guess from that, that you will cause the machine not to be very fast with that many folders (if they are below one main folder ,you will cause it surely to hang for awhile, while accessing the structure).
Just with a short view to your 500,000 numbers, i'm sure that whatever you want to do can be done better than creating so many folders. Like having a Database or Just have some information compressed together etc. etc.
If you explain what you are planning to create, we maybe can give you some better solutions than your mass-folder-project
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.