[issue14455] plistlib unable to read json and binary plist files
New submission from d9pouces :
Hi,
Plist files have actually three flavors : XML ones, binary ones, and now
(starting from Mac OS X 10.7 Lion) json one. The plistlib.readPlist function
can only read XML plist files and thus cannot read binary and json ones.
The binary format is open and described by Apple
(http://opensource.apple.com/source/CF/CF-550/CFBinaryPList.c).
Here is the diff (from Python 2.7 implementation of plistlib) to transparently
read both binary and json formats.
API of plistlib remains unchanged, since format detection is done by
plistlib.readPlist.
An InvalidFileException is raised in case of malformed binary file.
57,58c57
< "Plist", "Data", "Dict",
< "InvalidFileException",
---
> "Plist", "Data", "Dict"
64d62
< import json
66d63
< import os
68d64
< import struct
81,89c77,78
< header = pathOrFile.read(8)
< pathOrFile.seek(0)
< if header == ' p = PlistParser()
> rootObject = p.parse(pathOrFile)
195,285d183
<
< # timestamp 0 of binary plists corresponds to 1/1/2001 (year of Mac OS X
10.0), instead of 1/1/1970.
< MAC_OS_X_TIME_OFFSET = (31 * 365 + 8) * 86400
<
< class InvalidFileException(ValueError):
< def __str__(self):
< return "Invalid file"
< def __unicode__(self):
< return "Invalid file"
<
< def readBinaryPlistFile(in_file):
< """
< Read a binary plist file, following the description of the binary format:
http://opensource.apple.com/source/CF/CF-550/CFBinaryPList.c
< Raise InvalidFileException in case of error, otherwise return the root
object, as usual
< """
< in_file.seek(-32, os.SEEK_END)
< trailer = in_file.read(32)
< if len(trailer) != 32:
< return InvalidFileException()
< offset_size, ref_size, num_objects, top_object, offset_table_offset =
struct.unpack('>6xBB4xL4xL4xL', trailer)
< in_file.seek(offset_table_offset)
< object_offsets = []
< offset_format = '>' + {1: 'B', 2: 'H', 4: 'L', 8: 'Q', }[offset_size] *
num_objects
< ref_format = {1: 'B', 2: 'H', 4: 'L', 8: 'Q', }[ref_size]
< int_format = {0: (1, '>B'), 1: (2, '>H'), 2: (4, '>L'), 3: (8, '>Q'), }
< object_offsets = struct.unpack(offset_format, in_file.read(offset_size *
num_objects))
< def getSize(token_l):
< """ return the size of the next object."""
< if token_l == 0xF:
< m = ord(in_file.read(1)) & 0x3
< s, f = int_format[m]
< return struct.unpack(f, in_file.read(s))[0]
< return token_l
< def readNextObject(offset):
< """ read the object at offset. May recursively read sub-objects
(content of an array/dict/set) """
< in_file.seek(offset)
< token = in_file.read(1)
< token_h, token_l = ord(token) & 0xF0, ord(token) & 0x0F #high and low
parts
< if token == '\x00':
< return None
< elif token == '\x08':
< return False
< elif token == '\x09':
< return True
< elif token == '\x0f':
< return ''
< elif token_h == 0x10: #int
< result = 0
< for k in xrange((2 << token_l) - 1):
< result = (result << 8) + ord(in_file.read(1))
< return result
< elif token_h == 0x20: #real
< if token_l == 2:
< return struct.unpack('>f', in_file.read(4))[0]
< elif token_l == 3:
< return struct.unpack('>d', in_file.read(8))[0]
< elif token_h == 0x30: #date
< f = struct.unpack('>d', in_file.read(8))[0]
< return datetime.datetime.utcfromtimestamp(f +
MAC_OS_X_TIME_OFFSET)
< elif token_h == 0x80: #data
< s = getSize(token_l)
< return in_file.read(s)
< elif token_h == 0x50: #ascii string
< s = getSize(token_l)
< return in_file.read(s)
< elif token_h == 0x60: #unicode string
< s = getSize(token_l)
< return in_file.read(s * 2).decode('utf-16be')
< elif token_h == 0x80: #uid
< return in_file.read(token_l + 1)
< elif token_h == 0xA0: #array
< s = getSize(token_l)
< obj_refs = struct.unpack('>' + ref_format * s, in_file.read(s
[issue14455] plistlib unable to read json and binary plist files
d9pouces added the comment: Here is the new patch. I assumed that you meant to use diff -c instead of the raw diff command. -- keywords: +patch Added file: http://bugs.python.org/file25076/context.diff ___ Python tracker <http://bugs.python.org/issue14455> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14455] plistlib unable to read json and binary plist files
d9pouces added the comment: storchaka > I'm trying to take care of your remarks. So, I'm working on a more object-oriented code, with both write and read functions. I just need to write some test cases. IMHO, we should add a new parameter to the writePlist function, to allow the use of the binary or the json format of plist files instead of the default XML one. -- ___ Python tracker <http://bugs.python.org/issue14455> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14455] plistlib unable to read json and binary plist files
d9pouces added the comment:
I'm working on a class, BinaryPlistParser, which allow to both read and write
binary files.
I've also added a parameter fmt to writePlist and readPlist, to specify the
format ('json', 'xml1' or 'binary1', using XML by default). These constants are
used by Apple for its plutil program.
I'm now working on integrating these three formats to the test_plistlib.py.
However, the json is less expressive than the other two, since it cannot handle
dates.
--
___
Python tracker
<http://bugs.python.org/issue14455>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14455] plistlib unable to read json and binary plist files
d9pouces added the comment: Here is the new patch, allowing read and write binary, json and xml plist files. It includes both the plistlib.py and test/test_plistlib.py patches. JSON format does not allow dates and data, so XML is used by default to write files. I use the json library to write JSON plist files, but its output is slightly different from the Apple default output: keys of dictionaries are in different order. Thus, I removed the test_appleformattingfromliteral test for JSON files. Similarly, my binary writer does not write the same binary files as the Apple library: my library writes the content of compound objects (dicts, lists and sets) before the object itself, while Apple writes the object before its content. Copying the Apple behavior results in some additional weird lines of code, for little benefit. Thus, I also removed the test_appleformattingfromliteral test for binary files. Other tests are made for all the three formats. -- Added file: http://bugs.python.org/file25156/plistlib_with_test.diff ___ Python tracker <http://bugs.python.org/issue14455> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14455] plistlib unable to read json and binary plist files
d9pouces added the comment: The plutil (Apple's command-line tool to convert plist files from a format to another) returns an error if you try to convert a XML plist with dates to JSON. -- ___ Python tracker <http://bugs.python.org/issue14455> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14455] plistlib unable to read json and binary plist files
d9pouces added the comment: I just signed this agreement. Thanks for accepting this patch! -- ___ Python tracker <http://bugs.python.org/issue14455> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
