Re: $RANDOM not Cryptographically secure pseudorandom number generator

[Ole Tange]

On Sun, Dec 16, 2018 at 6:41 AM Eduardo Bustamante  wrote:
:
> You know no one is stopping you from submitting a patch to actually
> fix the documentation right? (or maybe, you know, submitting an actual
> working patch to change the random generator, not just drop some
> irrelevant code snippet you got from Wikipedia).

Patch attached.

It is basically a copy of the code snippet from Wikipedia with a few
trivial wrappers.

Apart from using Salsa20 the biggest change is that you can now seed
RANDOM with a string.


/Ole
diff --git a/doc/bash.1 b/doc/bash.1
index 9a7a384a..4c4d3882 100644
--- a/doc/bash.1
+++ b/doc/bash.1
@@ -1825,11 +1825,16 @@ generated.  The sequence of random numbers may be initialized by assigning
 a value to
 .SM
 .BR RANDOM .
+The value can be a string.
 If
 .SM
 .B RANDOM
 is unset, it loses its special properties, even if it is
 subsequently reset.
+.SM
+.B RANDOM
+uses Salsa20 for generating the integers, which as of 2015 has
+no published cryptanalysis attacks.
 .TP
 .B READLINE_LINE
 The contents of the
diff --git a/variables.c b/variables.c
index be2446e0..a4a7af52 100644
--- a/variables.c
+++ b/variables.c
@@ -18,6 +18,7 @@
along with Bash.  If not, see .
 */
 
+#include 
 #include "config.h"
 
 #include "bashtypes.h"
@@ -1295,53 +1296,125 @@ init_seconds_var ()
   return v;  
 }
  
-/* The random number seed.  You can change this by setting RANDOM. */
-static unsigned long rseed = 1;
-static int last_random_value;
 static int seeded_subshell = 0;
 
-/* A linear congruential random number generator based on the example
-   one in the ANSI C standard.  This one isn't very good, but a more
-   complicated one is overkill. */
+/* Salsa20 Cryptographically secure pseudorandom number generator from
+   https://en.wikipedia.org/wiki/Salsa20 */
+
+#define ROTL(a,b) (((a) << (b)) | ((a) >> (32 - (b
+#define QR(a, b, c, d)(		\
+	b ^= ROTL(a + d, 7),	\
+	c ^= ROTL(b + a, 9),	\
+	d ^= ROTL(c + b,13),	\
+	a ^= ROTL(d + c,18))
+#define ROUNDS 20
+
+uint32_t salsastate[16] = {
+  /* Start value: "expand 32-byte k" */
+  'e', 'x', 'p', 'a', 'n', 'd', ' ', '3', '2', '-', 'b', 'y', 't', 'e', ' ', 'k'
+};
+
+void
+salsa20_block ()
+{
+  int i;
+  uint32_t x[16];
+
+  for (i = 0; i < 16; ++i)
+x[i] = salsastate[i];
+  // 10 loops × 2 rounds/loop = 20 rounds
+  for (i = 0; i < ROUNDS; i += 2) {
+// Odd round
+QR(x[ 0], x[ 4], x[ 8], x[12]);	// column 1
+QR(x[ 5], x[ 9], x[13], x[ 1]);	// column 2
+QR(x[10], x[14], x[ 2], x[ 6]);	// column 3
+QR(x[15], x[ 3], x[ 7], x[11]);	// column 4
+// Even round
+QR(x[ 0], x[ 1], x[ 2], x[ 3]);	// row 1
+QR(x[ 5], x[ 6], x[ 7], x[ 4]);	// row 2
+QR(x[10], x[11], x[ 8], x[ 9]);	// row 3
+QR(x[15], x[12], x[13], x[14]);	// row 4
+  }
+  for (i = 0; i < 16; ++i)
+salsastate[i] += x[i];
+}
+
+int state_idx = 0;
 
-/* Returns a pseudo-random number between 0 and 32767. */
 static int
 brand ()
 {
-  /* From "Random number generators: good ones are hard to find",
- Park and Miller, Communications of the ACM, vol. 31, no. 10,
- October 1988, p. 1195. filtered through FreeBSD */
-  long h, l;
-
-  /* Can't seed with 0. */
-  if (rseed == 0)
-rseed = 123459876;
-  h = rseed / 127773;
-  l = rseed % 127773;
-  rseed = 16807 * l - 2836 * h;
-#if 0
-  if (rseed < 0)
-rseed += 0x7fff;
-#endif
-  return ((unsigned int)(rseed & 32767));	/* was % 32768 */
+  /* salsa20_block generates 16 values */
+  /* use them all before calling salsa20_block again */
+  if(0 == state_idx) {
+salsa20_block ();
+  }
+  state_idx++;
+  /* Keep 0 <= state_idx < 16 */
+  state_idx &= 0xF;
+  return ((unsigned int)(salsastate[state_idx] & 32767));
 }
 
-/* Set the random number generator seed to SEED. */
+/* set salsastate to 0 */
+static void
+reset_salsastate ()
+{
+  int i;
+  for (i = 0; i < 16; ++i)
+salsastate[i] = 0;
+}
+
+/* backwards compatible 32-bit seeder */
 static void
 sbrand (seed)
  unsigned long seed;
 {
-  rseed = seed;
-  last_random_value = 0;
+  int i;
+  reset_salsastate ();
+  salsastate[0] = seed & 0x;
+  state_idx = 0;
+}
+
+/* add seed with any binary input e.g. a string */
+/* similar to adding entropy to /dev/random */
+static void
+stringaddseedrand (seed,len)
+ uint8_t *seed;
+ int len;
+{
+  int i;
+  for (i = 0; i < len; ++i) {
+salsastate[i%16] ^= seed[i];
+if (i%16 == 15)
+  /* shake the bits before adding more */
+  /* otherwise 16 bytes repeated twice will cancel out */
+  salsa20_block ();
+  }
+  salsa20_block ();
+}
+
+/* seed with binary input of any size e.g. a string */
+static void
+stringseedrand (seed,len)
+ uint8_t *seed;
+ int len;
+{
+  reset_salsastate ();
+  stringaddseedrand (seed,len);
+  state_idx = 0;
 }
 
 static void
 seedrand ()
 {
+  /* add some semi-random seed */
+  /* look at https://github.com/jirka-h/haveged for better input */
   struct timeval tv;
-
+  uint32_t p

Re: Should [[ -v 1 ]] be supported?

[Greg Wooledge]

On Thu, Dec 27, 2018 at 08:57:12PM -0500, G. Branden Robinson wrote:
> What is it you're trying to achieve?  State your goal in terms that are
> "SMART":
> 
> (S)pecific
> (M)easurable
> (A)ttainable
> (R)easonable
> (T)imely

For the record, I tried for approximately two years to get Peng Yu
to state his goals, in a multitude of bug-bash and help-bash threads.
I tried the carrot, and I tried the stick.  Not ONCE did he ever do so.
I hope you have better luck than I did, but I wouldn't count on it.

Re: Should [[ -v 1 ]] be supported?

[Peng Yu]

> A profiler is exactly what you need here. You should profile your
> script and understand the stuff that actually matters for your goals.
> Otherwise you're just chasing unimportant things.

Again, my goal is not to profile a specific bash script. The goal is
to see what features make bash only fit into a shell language but
cannot make into other domains that other languages (e.g. python) are
popular at.

-- 
Regards,
Peng

Re: Should [[ -v 1 ]] be supported?

[Chet Ramey]

On 12/27/18 7:24 PM, Peng Yu wrote:
>> I don't believe that at all. The number of positional parameters is kept
>> anyway. It's not recalculated when you compare it to another number, so
>> it's just as fast as a simple comparison of two integers.
> 
> Getting the number $# is slow.

You haven't presented any evidence to support this, nor any basis for
comparison.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Should [[ -v 1 ]] be supported?

[Chet Ramey]

On 12/27/18 3:11 PM, Martijn Dekker wrote:

> Consistency might be a better argument. If [[ -v foo ]] is equivalent to [[
> -n ${foo+s} ]] for variables (with the advantage that you don't need 'eval'
> to handle arbitrary values of 'foo'), then perhaps it's not unreasonable to
> expect [[ -v 1 ]] to be equivalent to [[ -n ${1+s} ]].

The completeness argument is more rigorous, and there's a case to add this
in a future version of bash. He didn't make that argument, though.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Should [[ -v 1 ]] be supported?

[Chet Ramey]

On 12/27/18 11:32 PM, Peng Yu wrote:
>> You're whacking moles.  Use a profiler.  That's what they're for.
> 
> I've already shown that $() is a major problem to slow down the speed
> and I have reduced using its usage in my code and significantly
> improved the performance. 

This isn't a major deductive leap, since command substitution forks a
child and reads through a pipe. Forking is always more expensive than
not.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: How to compile hashlib.c for testing?

[Chet Ramey]

On 12/27/18 6:58 PM, Peng Yu wrote:

>> So you need a definition for xmalloc. The easiest thing to do is to add
>> an xmalloc function in the TEST_HASHING block, or you could follow the
>> directions in the comment there.
>>
>> Sooner or later, you're going to have to pick this stuff up on your own.
> 
> 
> I run make in lib/malloc, then I got the following error. What is wrong?

I encourage you to do the "easiest thing" I recommended above.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: How to compile hashlib.c for testing?

[Chet Ramey]

On 12/27/18 6:58 PM, Peng Yu wrote:

> Since the main() function is already there, why there is not already
> an easy way to compile it? How do you do unit-testing then for the
> code?

That code hasn't really changed in almost twenty years. All the testing
was done long ago.

(I just changed it recently to add comments and modify the FNV-1 offset,
but that doesn't change the functionality.)

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: readline fails to understand options.

[Bize Ma]

>
> Sure, it's a candidate for inclusion in a future version. I wouldn't object
> if someone wanted to do a sample implementation, since it changes how the
> variable value is parsed.
>

I only wish I knew enough C to make any (even small) positive
recommendation.

Re: write() not retried after EINTR in printf and echo

[Bize Ma]

Eduardo Bustamante () wrote:

>
> (...)



> What if instead of
> complaining you do something about it, like, fixing the problem (send
> a patch)?
>

You are assuming that if I take a look at the c code I will be able to read
it,
then understand it, and then make meaningful, or even reasonable changes
to it. My experience tells me I can not.

Not even read C correctly.

So, sadly I can not send a patch, even if I were to try …

Chet doesn't owe you anything.
>

Of course he doesn't owe me anything.
As much as I don't owe *you* anything.

But as I am reading your complaint to me and I am giving you an answer,
I expect that Chet could be able to read a complaint, take the good parts,
if any, and answer accordingly.

Re: Unexpected delay in using arguments.

[Bize Ma]

Chet Ramey () wrote:

> On 12/23/18 12:07 PM, Bize Ma wrote:


{...}


> > Have you been able to "take a look" ?
>
> Yes, as a matter of fact. Look at the changelog for the day or two around
> the date of the referenced messages.
>

I am pretty sure that I will not understand the finer details of the c code
even
if I could take a look at it. But, for the learning: could you be so kind
as to
provide some pointer as to where to find the change log ? Thanks!.

Re: Error on arithmetic evaluation of `~0`.

[Bize Ma]

Chet Ramey () wrote:

> On 12/23/18 12:01 PM, Bize Ma wrote:
>
{…}

> > Both command line above should have printed "hello".
>
> No. 0 is the only valid subscript for a non-array variable. The difference
> between bash and other shells that implement this feature is that bash
> warns about negative subscripts.
>

If you say so: fine for me.

It still irks me a little that a `${var[-1]}` isn't the "last value"
(sometimes!, consistency?).

I haven't seen that documented anywhere, though.

[IDEA] more granular shell options to fix errexit

[Ivan Pozdeev]

With the recent inherit_errexit , the Bash team seems to have finally 
bumped into a workable way to fix errexit which has been broken for 
decades (https://mywiki.wooledge.org/BashFAQ/105).


Instead of trying to invent a replacement shell option for that and hope 
to get it right on the first try (that errexit failed to), make granular 
options that would each change a single thing -- and see which of them 
pass the test of time.


The following suggested options aim to eliminate all cases that still 
exist, outlined on the above link, where an exit code of a command is 
swallowed/ignored.


* Do not unset `-e' inside a "checked command": only use the logic in 
the "The shell does not exit" clause when checking the exit code of the 
checked command, as a unit.


   (I would argue that this is the intended POSIX semantic in the first 
place but I guess I'm about 25 years late to the party.)

   Suggested option name: preserve_errexit

* If a command substitution fails, fail the containing command. 
Suggested name: cmdsubstfail


* Same for process substitution. Fail the command the same way as a 
pipeline with pipefail (and either only if pipefail is set, or imply 
pipefail). Suggested name: prsubstfail


(Probably the same for history expansion. I don't use it so have no 
opinion here.)


I don't have an opinion if they should be shell options or `shopt' 
options -- no differences to speak of in my book.


--
Regards,
Ivan