Pattern Specific Variable Limitation

[Matthew Von-Maszewski]

Dear GNU,

I found a small, but in my case defeating, limitation with the Pattern
Specific Variable code.  The code finds the first match for a pattern, not
necessarily the most specific match.  Example:

akamai/% : CFLAGS = -DSHORT
akamai/utils/% : CFLAGS = -DLONG

Any rule with a target beginning with "akamai/" will match the first
pattern, including all targets in the "akamai/utils/" directory.  So all
targets with a path of "akamai/*" and "akamai/utils/*" get CFLAGS = -DSHORT.
This pretty much blew all of today.

[Note:  our usage has the two lines being defined separately in makefiles
that get included into a master make.  It is not possible to easily
guarantee the order of the lines since the user may start from any point in
the directory hierarchy.]

Below is the code I applied to rule.c in make-3.79.1.

Please let me know if you believe this to be a properly formed patch.
Normally I am a Windows programmer, so I might not have this too pretty.
The work was performed on a Linux box (config.h attached).


Diff:
=
matthewv@a172-24-251-5 make-3.79.1]$ diff rule.c rule.org
588,592c588,589
<   struct pattern_var *p, *best;
<   unsigned int bestlen, targlen = strlen(target);
<
<   best=0;
<   bestlen=0;
---
>   struct pattern_var *p;
>   unsigned int targlen = strlen(target);
618,623c615,616
<   && (*p->suffix == '\0' || streq (&p->suffix[1],
&stem[stemlen+1]))
< && p->len >= bestlen)
<   {
< best=p;
< bestlen=p->len;
<   }
---
>   && (*p->suffix == '\0' || streq (&p->suffix[1],
&stem[stemlen+1])))
>     break;
626c619
<   return best;
---
>   return p;
[matthewv@a172-24-251-5 make-3.79.1]$

=
Matthew Von-Maszewski
 Director of Process and Software Quality Engineering

 work: 617-250-4669 - [EMAIL PROTECTED]
pager: 888-402-0573 - [EMAIL PROTECTED]
 cell: 617-780-5135

 config.h

RE: Pattern Specific Variable Limitation

[Matthew Von-Maszewski]

Hmm,

I understand your concern.  What is the impact to older makefiles if this
change is implemented?  I would assess the impact as follows:

1. User worked around the bug:  by changing the order of the rules, a user
can get around the limitation.  These users would not be effected by this
change.

2. User unaware that more specific patterns are "filtered" by less specific
patterns:  Wow.  This could be interesting.  This suggests they either have
code that is building wrong, or could unmask older statements that never
worked (hence, never corrected).


I did stop and re-read the gnu make documentation on patterns.  No where
does it explicitly say how the rules would be applied in this case.  At
minimum, I recommend you update the documentation with whatever decision you
make.  I also reread the IEEE Posix description.  Pattern rules do not
appear in the standard I have.  Again the issue is ambiguous.

Arguments:
- for sequence:  maintain historical characteristics, prerequisites get
processed in sequence, simply expanded variables are processed in sequence

- for best match:  would create more flexibility for included makefiles (my
problem), rules are processed in order of dependency (not sequence),
recursively expanded variables are processed at time of need (not sequence
of assignment)

>From a safety and compatibility standpoint, I would agree to not change.

>From the standpoint of "all that makes make special", I would argue for the
change.  I never think of a makefile as a sequential piece of code.  That is
what I love.  I think of the makefile as a collection of rules, in any
order, that the make tool then resolves into the correct order based upon
the current conditions.  This why dual processor mode (-j2) works so
beautifully (just think about scripting a dual processor mode shell script
...).  The current implementation implies that order is now essential in a
tool that is based upon being order independent.

Thanks for your consideration,
Matthew

-Original Message-
From: Paul D. Smith [mailto:[EMAIL PROTECTED]]On Behalf Of
Paul D. Smith
Sent: Sunday, March 18, 2001 10:35 PM
To: Matthew Von-Maszewski
Cc: [EMAIL PROTECTED]
Subject: Re: Pattern Specific Variable Limitation


%% "Matthew Von-Maszewski" <[EMAIL PROTECTED]> writes:

  mv> I found a small, but in my case defeating, limitation with the
  mv> Pattern Specific Variable code.  The code finds the first match
  mv> for a pattern, not necessarily the most specific match.

I'll have to think about this change.  In all the other pattern rules in
make, the first appropriate match always wins; nowhere else in make
does it do any kind of "best fit" match such as you are proposing here.

  mv> Please let me know if you believe this to be a properly formed
  mv> patch.

Actually, not :).  It _would_ help if you could provide a proper patch.

First, the order of arguments to diff is "diff  "; you have
used "diff rule.c rul.org" which I interpret to be "diff  ",
so your diff is backwards (showing lines removed that were really added
and added that were really removed).

Also, any diff created for the purpose of patch should use either the -c
(context) or -u (unified) diff format; these forms include 3 lines of
unchanged text around the actual changes which allows patch to be much
more accurate about finding the right chunk of code to apply the changes
to.  I have a preference for -u, others prefer -c; it doesn't really
matter.

Thanks.

--

---
 Paul D. Smith <[EMAIL PROTECTED]>  Find some GNU make tips at:
 http://www.gnu.org  http://www.paulandlesley.org/gmake/
 "Please remain calm...I may be mad, but I am a professional." --Mad
Scientist


___
Bug-make mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/bug-make

RE: Pattern Specific Variable Limitation

[Matthew Von-Maszewski]

Yep, my case is light against the history of the tool.

You and I could argue all day about interpretations of "equally applicable"
and "order in which pattern rules appear".  You read this a firm requirement
definition.  I read it as fuzzy.  But the bottom line is what is in the
code.

I do appreciate your time spent in reviewing and discussing this change.

Thanks,
Matthew


-Original Message-
From: Paul D. Smith [mailto:[EMAIL PROTECTED]]On Behalf Of
Paul D. Smith
Sent: Monday, March 19, 2001 9:50 AM
To: Matthew Von-Maszewski
Cc: [EMAIL PROTECTED]
Subject: RE: Pattern Specific Variable Limitation


%% "Matthew Von-Maszewski" <[EMAIL PROTECTED]> writes:

  mv> I understand your concern.  What is the impact to older makefiles
  mv> if this change is implemented?

There are more issues than this, too:

 1) Difference in behavior with other pattern rules.  The way "normal"
pattern rules are handled well-known and well-documented in the
manual: the rules are processed in order of definition, and each one
is tried in that order.

Since target-specific patterns look and feel like "normal" patterns,
having them behave differently in this respect is not something to
take lightly.

 2) Flexibility: if you keep an order definition then people can do what
they want either way by changing the order.  If the change you
suggest is made, then there is no way to get the "old" behavior back
again, if that's what you want.

  mv> I did stop and re-read the gnu make documentation on patterns.

The GNU make manual says:

 >The order in which pattern rules appear in the makefile is important
 > since this is the order in which they are considered.  Of equally
 > applicable rules, only the first one found is used.

("Introduction to Pattern Rules").  By "equally applicable" here they
mean "rules whose targets match the pattern, and whose prerequisites can
be created".

Obviously this is talking about pattern rules themselves, not
target-specific pattern rules, but my point is that having different
behavior between them is potentially confusing.

  mv> - for best match: would create more flexibility for included
  mv>   makefiles (my problem), rules are processed in order of
  mv>   dependency (not sequence), recursively expanded variables are
  mv>   processed at time of need (not sequence of assignment)

I don't understand your last point here?

  mv> From the standpoint of "all that makes make special", I would
  mv> argue for the change.  I never think of a makefile as a sequential
  mv> piece of code.

This view of makefiles as non-programmatic is not _entirely_ correct,
though.  Certainly when walking the dependency graph the ordering has
nothing to do with the makefile, as it should be.  However, many parts
of make _do_ depend on the order of definition in the makefile.  I
quoted the manual above, and there are variable definitions, etc. as
well.

--

---
 Paul D. Smith <[EMAIL PROTECTED]>  Find some GNU make tips at:
 http://www.gnu.org  http://www.paulandlesley.org/gmake/
 "Please remain calm...I may be mad, but I am a professional." --Mad
Scientist


___
Bug-make mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/bug-make

RE: Pattern Specific Variable Limitation

[Matthew Von-Maszewski]

We are in agreement.  I do not like it, but I do understand.

Thanks again for the time,
Matthew


-Original Message-
From: Paul D. Smith [mailto:[EMAIL PROTECTED]]On Behalf Of
Paul D. Smith
Sent: Monday, March 19, 2001 11:09 AM
To: Matthew Von-Maszewski
Cc: [EMAIL PROTECTED]
Subject: RE: Pattern Specific Variable Limitation


%% "Matthew Von-Maszewski" <[EMAIL PROTECTED]> writes:

  mv> You and I could argue all day about interpretations of "equally
  mv> applicable" and "order in which pattern rules appear".  You read
  mv> this a firm requirement definition.  I read it as fuzzy.  But the
  mv> bottom line is what is in the code.

Well, I think "order in which pattern rules appear in the makefile" is
pretty unambiguous.  I do agree that "equally applicable rules" could be
somewhat open to interpretation, and perhaps should be clarified.

However, this cannot be changed: I know for a fact that there are many
makefiles depending on this ordering behavior for pattern rules.

The pattern rules ordering for target-specific patterns is less set,
since they're so new, so it's not as clear-cut.  However, as I mentioned
before, I think it would in general be confusing if pattern rules and
target-specific patterns used different matching methods.

--

---
 Paul D. Smith <[EMAIL PROTECTED]>  Find some GNU make tips at:
 http://www.gnu.org  http://www.paulandlesley.org/gmake/
 "Please remain calm...I may be mad, but I am a professional." --Mad
Scientist


___
Bug-make mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/bug-make

Win32 File Name Compares and Normalization

[Matthew Von-Maszewski]

Greetings,

I have been doing extensive work with GNU make and Microsoft's compiler.
This includes creating a dependency generator similar the gcc's .d files.
It has become obvious that a couple of the make's functions need to become
case insensitive when on a Win 32 platform.  Microsoft's file names are
internally considered to be case insensitive for its file operations and
matching.

I have listed the changes below that I found sufficient to create normalized
filenames (all lowercase) and to remove case sensitive compares (this needs
a cleaner implementation).

[My dependency generator depends upon a preprocessor dump from Microsoft's
compiler.  Depending upon the command line options, sometimes the
preprocessor normalizes all file names to lower case (e.g. /ZI option).
Other times case is left as found in the file system.  Lovely.]

All changes are against the version 3.79.1 download.

Would you please consider this change request?

Thank you,
Matthew

Index: glob/fnmatch.c
===
RCS file: /home/cvsroot/gnumake/glob/fnmatch.c,v
retrieving revision 1.1.1.1
retrieving revision 1.2
diff -c -r1.1.1.1 -r1.2
*** glob/fnmatch.c  2001/09/10 15:33:48 1.1.1.1
--- glob/fnmatch.c  2001/09/10 15:37:35 1.2
***
*** 168,174 
  # ifdef _LIBC
  #  define FOLD(c) ((flags & FNM_CASEFOLD) ? tolower (c) : (c))
  # else
! #  define FOLD(c) ((flags & FNM_CASEFOLD) && ISUPPER (c) ? tolower (c) :
(c))
  # endif

while ((c = *p++) != '\0')
--- 168,176 
  # ifdef _LIBC
  #  define FOLD(c) ((flags & FNM_CASEFOLD) ? tolower (c) : (c))
  # else
! /*#  define FOLD(c) ((flags & FNM_CASEFOLD) && ISUPPER (c) ? tolower (c) :
(c)
)*/
! /* mev */
! #  define FOLD(c) ( ISUPPER (c) ? tolower (c) : (c))
  # endif


Index: w32/pathstuff.c
===
RCS file: /home/cvsroot/gnumake/w32/pathstuff.c,v
retrieving revision 1.1.1.1
retrieving revision 1.2
diff -c -r1.1.1.1 -r1.2
*** w32/pathstuff.c 2001/09/10 15:33:48 1.1.1.1
--- w32/pathstuff.c 2001/09/10 15:37:36 1.2
***
*** 80,85 
--- 80,87 
  if (*p == '\\')
  *p = '/';

+ /* mev */
+ strlwr(w32_path);
  return w32_path;
  }

cvs server: Diffing w32/compat
Index: w32/compat/dirent.c
===
RCS file: /home/cvsroot/gnumake/w32/compat/dirent.c,v
retrieving revision 1.1.1.1
retrieving revision 1.2
diff -c -r1.1.1.1 -r1.2
*** w32/compat/dirent.c 2001/09/10 15:33:48 1.1.1.1
--- w32/compat/dirent.c 2001/09/10 15:37:36 1.2
***
*** 116,122 
/* fill in struct dirent values */
pDir->dir_sdReturn.d_ino = -1;
strcpy(pDir->dir_sdReturn.d_name, wfdFindData.cFileName);
!
return &pDir->dir_sdReturn;
  }

--- 116,123 
/* fill in struct dirent values */
pDir->dir_sdReturn.d_ino = -1;
strcpy(pDir->dir_sdReturn.d_name, wfdFindData.cFileName);
! /* mev */
! strlwr(pDir->dir_sdReturn.d_name);
return &pDir->dir_sdReturn;
  }



==
Matthew Von-Maszewski
 [EMAIL PROTECTED]
 508-870-0118



___
Bug-make mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/bug-make