Re: regular expression problem

[Karsten Hilbert]

On Sun, Oct 28, 2018 at 11:14:15PM +, MRAB wrote:

> > - lines can contain several placeholders
> > 
> > - placeholders start and end with '$'
> > 
> > - placeholders are parsed in three passes
> > 
> > - the pass in which a placeholder is parsed is denoted by the number of '<' 
> > and '>' next to the '$':
> > 
> > $<...>$ / $<<...>>$ / $<<<...>>>$
> > 
> > - placeholders for different parsing passes must be nestable:
> > 
> > $<<<...$<...>$...>>>$
> > 
> > (lower=earlier parsing passes will be inside)
> > 
> > - the internal structure is "name::options::range"
> > 
> > $$
> > 
> > - name will *not* contain '$' '<' '>' ':'
> > 
> > - range can be either a length or a "from-until"
> > 
> > - a length will be a positive integer (no bounds checking)
> > 
> > - "from-until" is: a positive integer, a '-', and a positive integer (no 
> > sanity checking)
> > 
> > - options needs to be able to contain nearly anything, except '::'
> > 
> > 
> > Is that sufficiently defined and helpful to design the regular expression ?
> > 
> How can they be nested inside one another?
> Is the string scanned, placeholders filled in for that level, and then the
> string scanned again for the next level? (That would mean that the fill
> value itself will be scanned in the next pass.)

Exactly. But *different* levels can be nested inside each other.

> You could try matching the top level, for each match then match the next
> level, and for each of those matches then match for the final level.

So I do.

> Trying to do it all in one regex is usually a bad idea.

Right, I am not trying to do that. I was, however, worried
that I need to make the expression not "trip over" fragments
of what might seem to constitute part of another placeholder.

$<$::15>>$

Pass 1 might fill in to:

$>$

and I was worried to make sure the second pass does not stop here:

$>$
   ^

Logically it should not because

>s'::15>>$

does not match

::\d*>>$

but I am not sure how to tell it that :-)

Karsten
-- 
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: regular expression problem

[Karsten Hilbert]

On Mon, Oct 29, 2018 at 12:10:04AM +0100, Thomas Jollans wrote:

> On 28/10/2018 22:04, Karsten Hilbert wrote:
> > - options needs to be able to contain nearly anything, except '::'
> 
> Including > and $ ?

Unfortunately, it might. Even if I assume that earlier passes
are "inside", and thusly "filled in" before outer=later
passes happen the fillin value might still contain some of :>$.

The fillin value will NOT contain a newly generated, matching
placeholder definition, though.

Karsten
-- 
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: regular expression problem

[Karsten Hilbert]

> Right, I am not trying to do that. I was, however, worried
> that I need to make the expression not "trip over" fragments
> of what might seem to constitute part of another placeholder.
> 
>   $<$::15>>$
> 
> Pass 1 might fill in to:
> 
>   $>$
> 
> and I was worried to make sure the second pass does not stop here:
> 
>   $>$
   ^

Here, of course.

Karsten
-- 
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: regular expression problem

[Karsten Hilbert]

On Sun, Oct 28, 2018 at 11:57:48PM +0100, Brian Oney wrote:

> On Sun, 2018-10-28 at 22:04 +0100, Karsten Hilbert wrote:
> > [^<:]
> 
> Would a simple regex work?

This brought about the solution.

However, not this way:

> >>> import re
> >>> t = '$$'
> >>> re.findall('[^<>:$]+', t)
> ['name', 'options', 'range']

because I am not trying to parcel out the placeholder *parts*
(but rather the placeholders from a given line).

I eventually figured that denoting the parsing stages
differently made for easier matching. Rather than

$<>$
$<<>>$
$<<<>>>$

do this

$1<>1$
$2<>2$
$3<>3$

which makes it way less ambiguous, and more matchable:

regexen = [
r'\$1{0,1}<[^<].*?>1{0,1}\$',
r'\$2<[^<].*?>2\$',
r'\$3<[^<].*?>3\$'
]

The [^<] part ("the single < is NOT to be followed directly
by another <") is actually superfluous but does protect
against legacy document templates still having
$<<(<)...(>)>>$ in them.

$<>$ is still retained as an alias for $1<>1$ because there is
A LOT of them in existing document templates. It is
normalized explicitely inside Python before fillin values are
generated.

Karsten
-- 
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: regular expression problem

[MRAB]

On 2018-10-29 08:02, Karsten Hilbert wrote:

On Sun, Oct 28, 2018 at 11:14:15PM +, MRAB wrote:


> - lines can contain several placeholders
> 
> - placeholders start and end with '$'
> 
> - placeholders are parsed in three passes
> 
> - the pass in which a placeholder is parsed is denoted by the number of '<' and '>' next to the '$':
> 
> 	$<...>$ / $<<...>>$ / $<<<...>>>$
> 
> - placeholders for different parsing passes must be nestable:
> 
> 	$<<<...$<...>$...>>>$

>
>(lower=earlier parsing passes will be inside)
> 
> - the internal structure is "name::options::range"
> 
> 	$$
> 
> - name will *not* contain '$' '<' '>' ':'
> 
> - range can be either a length or a "from-until"
> 
> - a length will be a positive integer (no bounds checking)
> 
> - "from-until" is: a positive integer, a '-', and a positive integer (no sanity checking)
> 
> - options needs to be able to contain nearly anything, except '::'
> 
> 
> Is that sufficiently defined and helpful to design the regular expression ?
> 
How can they be nested inside one another?

Is the string scanned, placeholders filled in for that level, and then the
string scanned again for the next level? (That would mean that the fill
value itself will be scanned in the next pass.)


Exactly. But *different* levels can be nested inside each other.


You could try matching the top level, for each match then match the next
level, and for each of those matches then match for the final level.


So I do.


Trying to do it all in one regex is usually a bad idea.


Right, I am not trying to do that. I was, however, worried
that I need to make the expression not "trip over" fragments
of what might seem to constitute part of another placeholder.

$<$::15>>$

Pass 1 might fill in to:

$>$

and I was worried to make sure the second pass does not stop here:

$>$
^

Logically it should not because

>s'::15>>$

does not match

::\d*>>$

but I am not sure how to tell it that :-)


For something like that, I'd use parsing by recursive descent.

It might be worth looking at pyparsing.
--
https://mail.python.org/mailman/listinfo/python-list

Re: regular expression problem

[Karsten Hilbert]

On Mon, Oct 29, 2018 at 05:16:11PM +, MRAB wrote:

> > Logically it should not because
> > 
> > >s'::15>>$
> > 
> > does not match
> > 
> > ::\d*>>$
> > 
> > but I am not sure how to tell it that :-)
> > 
> For something like that, I'd use parsing by recursive descent.
> 
> It might be worth looking at pyparsing.

I feared as much. However, by slightly changing the boundary
conditions I was able to solve the problem :-)

Now I am only left with the task to search-replace a bunch of
LaTeX templates during the next database upgrade ...

Karsten
-- 
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
-- 
https://mail.python.org/mailman/listinfo/python-list

customer compare in assertEqual

[Stone Zhong]

Hi There,

Now I want to make sure my code calls a function foo with an object t, however, 
foo.assert_called_once_with(t) does not work, since t is a model object and the 
code may load a different copy of t, so what I really want to test is "It calls 
foo with t where t.id equals real_t.id, is there a way to do that in python 
unit test?

Thanks,
Stone
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: customer compare in assertEqual

[Stone Zhong]

On Monday, October 29, 2018 at 5:29:11 PM UTC-7, Stone Zhong wrote:
> Hi There,
> 
> Now I want to make sure my code calls a function foo with an object t, 
> however, foo.assert_called_once_with(t) does not work, since t is a model 
> object and the code may load a different copy of t, so what I really want to 
> test is "It calls foo with t where t.id equals real_t.id, is there a way to 
> do that in python unit test?
> 
> Thanks,
> Stone

Tried addTypeEqualityFunc and it does not seems help.

-- 
https://mail.python.org/mailman/listinfo/python-list