On 04/18/2011 04:32 AM, Doriano Blengino wrote: > Kevin Fishburne ha scritto: >> On 04/15/2011 03:56 AM, Doriano Blengino wrote: >> >>> Kevin Fishburne ha scritto: >>> >>>> I'm in the early phases of creating a "database" that uses the file >>>> system for data organization rather than a traditional software database >>>> such as MySQL,... >>>> >>>> I will have 4,194,304 "cells", each of which has about three datafiles >>>> that will need to be opened, read from, written to and closed regularly. >>>> ... >>>> I'm considering dividing them into hierarchies of directories to avoid >>>> having four to 16 million data files in the same directory. Initial >>>> tests hit file system (or file space, not sure yet) limits. >>>> >>>> >>>> >>> Very interesting problem. >>> >>> For the file system, the more sophisticated is, the more it is expensive >>> to modify it; etx3/4 or reiserfs are journaled, so they are slower and >>> heavier to manage. But it could be that a well-planned journaled fs is >>> faster than a bad-planned non-journaled fs... >>> >> Hi Doriano. Good advice about functionality versus speed. I'm going to >> be testing both ext2 and xfs this week to see which is superior for my >> purposes. As far as data integrity, I'll probably have some sort of >> local RAID as a backup target with slow, incremental writes to it. If >> the server dies, then at least most of the game data will be preserved >> without harming the performance of the server app. So the weakness of >> the filesystem with regard to crash recovery is irrelevant. >> > Please let us know about your tests...
I will. >> >> My current plan is to create a directory for each region >> ([65536/32/32]^2). Each region directory contains 32^2 data files >> (1024). Hopefully this won't stress any particular file system as far as >> how many directories and files are contained within a single directory. >> > But... I am missing something... the number of files was 4M, right? And > 64 directories with 1024 files does not sum up to 4M... A cell is 32x32 tiles (bytes), and the map is 65536^2 tiles, so there are 2048^2 cells. Each of these are organized into directories of 32^2 cells (regions). The data files alone are 4.2 million in number, the directories that organize them hopefully add a layer of filesystem efficiency. Right now a separate directory is created for each region and empty data files are created for each cell within that region. -- Kevin Fishburne Eight Virtues www: http://sales.eightvirtues.com e-mail: [email protected] phone: (770) 853-6271 ------------------------------------------------------------------------------ Benefiting from Server Virtualization: Beyond Initial Workload Consolidation -- Increasing the use of server virtualization is a top priority.Virtualization can reduce costs, simplify management, and improve application availability and disaster protection. Learn more about boosting the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev _______________________________________________ Gambas-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/gambas-user
