Re: [Development] utf-8 BOM and parsers

2014-04-23 Thread Thiago Macieira
Em qua 23 abr 2014, às 07:38:30, Thiago Macieira escreveu: > Em qua 23 abr 2014, às 08:14:41, Knoll Lars escreveu: > > >I'll send the update to the release branch in the next few hours. > > > > Thanks! > > I will do that today. I spent my Qt time yesterday with the changelog and > the header diff

Re: [Development] utf-8 BOM and parsers

2014-04-23 Thread Thiago Macieira
Em qua 23 abr 2014, às 08:14:41, Knoll Lars escreveu: > >I'll send the update to the release branch in the next few hours. > > Thanks! I will do that today. I spent my Qt time yesterday with the changelog and the header diff. Running the scripts took about 5 seconds for each Editing the changel

Re: [Development] utf-8 BOM and parsers

2014-04-23 Thread Knoll Lars
On 22/04/14 16:36, "Thiago Macieira" wrote: >Em ter 22 abr 2014, às 12:35:33, Knoll Lars escreveu: >> Hi, >> >> Just came back from vacation today. >> >> Unfortunately BOM’s at the beginning of files seem to still be used >>quite >> a bit esp. in the Windows world. So I would actually vote for

Re: [Development] utf-8 BOM and parsers

2014-04-22 Thread Thiago Macieira
Em ter 22 abr 2014, às 12:35:33, Knoll Lars escreveu: > Hi, > > Just came back from vacation today. > > Unfortunately BOM’s at the beginning of files seem to still be used quite > a bit esp. in the Windows world. So I would actually vote for option 1 and > rather keep compatibility. Reason is tha

Re: [Development] utf-8 BOM and parsers

2014-04-22 Thread Knoll Lars
Hi, Just came back from vacation today. Unfortunately BOM’s at the beginning of files seem to still be used quite a bit esp. in the Windows world. So I would actually vote for option 1 and rather keep compatibility. Reason is that stripping the BOM will not break anything, but leaving it in will.

Re: [Development] utf-8 BOM and parsers

2014-04-16 Thread Thiago Macieira
Em seg 14 abr 2014, às 10:33:48, Thiago Macieira escreveu: > Em seg 14 abr 2014, às 09:59:18, Thiago Macieira escreveu: > > Also, the Unix philosophy is that UTF-8 BOMs should not be used. This > > started on Windows, with tools like Notepad, where changing the system > > locale is not an option.

Re: [Development] utf-8 BOM and parsers

2014-04-16 Thread Friedemann Kleint
Hi, this is tracked by https://bugreports.qt-project.org/browse/QTBUG-37423 - keeping an eye on the release blocking bugs https://bugreports.qt-project.org/browse/QTBUG-37065 occasionally helps to minimize surprises ;-) . Friedemann -- Friedemann Kleint Digia, Qt ___

Re: [Development] utf-8 BOM and parsers

2014-04-16 Thread Rutledge Shawn
On 14 Apr 2014, at 2:26 PM, Simon Hausmann wrote: > We have various parsers in Qt that parse "source code" and do things with it, > such as the QML parser… I was just baffled by this issue this morning for a couple of hours: tried to "port" some QML code that was working fine with Qt 5.2.1 to

Re: [Development] utf-8 BOM and parsers

2014-04-15 Thread Allan Sandfeld Jensen
April 14, 2014 7:34 PM > > To: development@qt-project.org > > Subject: Re: [Development] utf-8 BOM and parsers > > Hi Thiago, > > Thanks for listening the reasons here in detail! > > > Em seg 14 abr 2014, às 09:59:18, Thiago Macieira escreveu: > > > Also, the U

Re: [Development] utf-8 BOM and parsers

2014-04-15 Thread Koehne Kai
Re: [Development] utf-8 BOM and parsers Hi Thiago, Thanks for listening the reasons here in detail! > Em seg 14 abr 2014, às 09:59:18, Thiago Macieira escreveu: > > Also, the Unix philosophy is that UTF-8 BOMs should not be used. This > > started on Windows, with tools like No

Re: [Development] utf-8 BOM and parsers

2014-04-14 Thread Thiago Macieira
Em seg 14 abr 2014, às 09:59:18, Thiago Macieira escreveu: > Also, the Unix philosophy is that UTF-8 BOMs should not be used. This > started on Windows, with tools like Notepad, where changing the system > locale is not an option. To be clear: BOMs are to be used to determine that the content *is

Re: [Development] utf-8 BOM and parsers

2014-04-14 Thread Thiago Macieira
Em seg 14 abr 2014, às 18:29:26, Olivier Goffart escreveu: > What were the reason to change that behaviour? > Personally, I think it's safer to keep the 5.2 behaviour and avoid breaking > user's code. It seemed wrong when I was rewriting that ours did like that. When I wrote tst_utf8 way back wh

Re: [Development] utf-8 BOM and parsers

2014-04-14 Thread Olivier Goffart
On Monday 14 April 2014 07:14:44 Thiago Macieira wrote: > Em seg 14 abr 2014, às 15:13:53, Frank Osterfeld escreveu: > > On 14 Apr 2014, at 14:26, Simon Hausmann wrote: > > > Since this affects not just one place but many (and for example we have > > > many copies of the QML lexer around), I'd lik

Re: [Development] utf-8 BOM and parsers

2014-04-14 Thread Thiago Macieira
Em seg 14 abr 2014, às 15:13:53, Frank Osterfeld escreveu: > On 14 Apr 2014, at 14:26, Simon Hausmann wrote: > > Since this affects not just one place but many (and for example we have > > many copies of the QML lexer around), I'd like to determine what the > > _correct_ fix for this issue is, bec

Re: [Development] utf-8 BOM and parsers

2014-04-14 Thread Frank Osterfeld
On 14 Apr 2014, at 14:26, Simon Hausmann wrote: > > Since this affects not just one place but many (and for example we have many > copies of the QML lexer around), I'd like to determine what the _correct_ fix > for this issue is, because frankly speaking I don't know :). However I have > an

[Development] utf-8 BOM and parsers

2014-04-14 Thread Simon Hausmann
Hi, We have various parsers in Qt that parse "source code" and do things with it, such as the QML parser, the CSS parser and others. We do make the assumption that their input is UTF-8 encoded and therefore have simply used QString code = QString::fromUtf8(byteArray); in some form or other