I've just had quite a struggle to package a medium sized database into
my application in a way that will populate that database on the phone
very quickly when the app first loads. But I'm victorious. I have read
several posts from the past about people having the same problems, but
they failed to post complete solutions. It would have saved me a lot of
time. Here is the unified solution (as I see it). I welcome any
input. I also have several questions at the end.
In the beginning, I wanted to simply pre-populate a database on android
using a plain text file containing a single word on each line. I placed
the file in the raw resource folder and set up my application to 1. set
up a database with and empty table, and then 2. use a
java.io.InputStreamReader to read line by line and run a SQL statement
inserting each line of data into the database. The application crashed
when I ran it and I got an error in my logcat:
D/asset (909): Data exceeds UNCOMPRESS_DATA_MAX (1424000 vs 1048576)
Turns out there is a file size limit of 1048576 bytes for resource files
in raw and also in assets. So since it was just a text file, I split it
into two files and ran the sequence twice, once on each file. It works
great and the database populates perfectly. HOWEVER, it takes 5 minutes
to insert 135,000 records. You can achieve slight speedup if you close
the whole series of inserts in a single SQL transaction (first issue
BEGIN TRANSACTION; then after the insert and update statements issue
COMMIT TRANSACTION;). If you have a small database, this solution will
work okay. But keep reading. There is a faster way.
I couldn't deal with a 5 minute startup time on an app, so I decided to
include the actual database file in my resource raw folder (others
suggested this to me). (You can pull your existing database off of your
emulator, if you happen to have gotten it on there as I did, using SQL
statements and text files, by copying it off the phone from
/data/data/com.yourdomain.applicationname/databases/applicationame.db
using android debug bridge, or by creating one using sqlite3 on your
desktop. Be sure to include the meta data table found in all android
databases.) Then in my code, I built a method to create an empty
database using the SQLiteOpenHelper--then immediately close it so I can
overwrite the file. Then the method tries to open the included db file
from raw resource as an input stream. But when I read from it, it thew
an exception every time. The reason? The db file was too large. This
took me forever to realize because the only exception was
java.io.IOException, which isn't very informative.
Finally, I decided to apply solution 1 again. That is, split the
files. I used the linux split command to split the binary database file
into a maximum of 1048576 bytes. The command is:
split infile -b 1048576 outfileprefix
and the files come out as outfileprefixaa, outfileprefixab, etc.
I copied these files into my raw resource folder and set up my code to
create and then close a database automatically using SQLiteOpenHelper
and then to run the following method:
private void copyDatabase() throws IOException{
OutputStream databaseOutputStream = new
FileOutputStream("/data/data/com.domain.app/databases/app.db");
InputStream databaseInputStream;
byte[] buffer = new byte[1024];
int length;
databaseInputStream =
databaseOpenHelperContext.getResources().openRawResource(R.raw.datafileaa);
while ( (length = databaseInputStream.read(buffer)) > 0 ) {
databaseOutputStream.write(buffer);
}
databaseInputStream.close();
databaseInputStream =
databaseOpenHelperContext.getResources().openRawResource(R.raw.datafileab);
while ( (length = databaseInputStream.read(buffer)) > 0 ) {
databaseOutputStream.write(buffer);
}
databaseInputStream.close();
databaseInputStream =
databaseOpenHelperContext.getResources().openRawResource(R.raw.datafileac);
while ( (length = databaseInputStream.read(buffer)) > 0 ) {
databaseOutputStream.write(buffer);
}
databaseInputStream.close();
databaseOutputStream.flush();
databaseOutputStream.close();
}
This solution works perfectly. And it's very fast. My final database
size on the emulator is 2714624 bytes (about 2.5 meg). It copies almost
instantly. There is no delay when the application first runs. However,
remember that the resource files remain installed, and you've copied the
data to a database. This effectively doubles the data storage usage of
the application.
My questions are:
1. Why is there a file size limit on resources in the raw or assets
folders (such a small limit, that is)?
2. Does this limit apply to all raw and asset files, or only to files
that you get an input stream for?
2. Can I change the file size limit, or is it a fixed feature?
3. Is it appropriate to split a raw file into two or three files and
read them in as three consecutive separate streams, all output to one
file? Or is this a bad practice since they must have implemented a file
size limit for storage conservation reasons?
4. Can I delete the resources from raw or assets at runtime (or is this
totally invalid since the compiler sets up R.raw to have methods
representing the files) to save space?
5. How big is too big an application? (mine is a large word dictionary
and the whole app after first run takes 4 meg)
I suppose a more space efficient model would be to offer the data online
somewhere, especially for bigger databases, and load from there, and
store the database on the SDCard, which requires you NOT use the
SQLiteOpenHelper.
Good luck to all you database pre-populators and I hope this helps. I
welcome any input and correction to this document as I am no expert and
may be doing something that isn't kosher in my code.
Justin Jaynes
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google
Groups "Android Developers" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/android-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
