Bug#636618: smaller example and some debugging

[jens]

I added some smaller example csv files: 2 columns seperated by dash
and 4 rows with the umlaut in line 3.

I debugged "column" and found that the problem is located in the
function input at line 291.
fgetws will only read two lines for the latin1 (iso88591) file.

I don't know if this is the correct behavior of fgetws if the locale
doesn't match the file encoding. I am no c programmer ;)
To get a complete convenient result, you would have to set the locale
to the file encoding before reading (unless it will brake at an
umlaut),
and to terminal encoding before printing (unless it will print strange
chars to you terminal).

In my test.c file i set the locale manually and in some combination
the file would be read completely.

$ gcc test.c && ./a.out

reading file: small_latin1.csv with locale: en_US.UTF-8
a0#b0
a1#b1

reading file: small_utf8.csv with locale: en_US.UTF-8
a0#b0
a1#b1
ä2#b2
a3#b3

reading file: small_latin1.csv with locale: de_DE.iso88591
a0#b0
a1#b1
ä2#b2
a3#b3

reading file: small_utf8.csv with locale: de_DE.iso88591
a0#b0
a1#b1
ä2#b2
a3#b3

reading file: small_latin1.csv with locale: en_US.UTF-8
a0#b0
a1#b1

reading file: small_utf8.csv with locale: en_US.UTF-8
a0#b0
a1#b1
ä2#b2
a3#b3

a0#b0
a1#b1
ä2#b2
a3#b3

a0#b0
a1#b1
ä2#b2
a3#b3

#include <wchar.h>
#include <stdio.h>
#include <locale.h>

#define	MAXLINELEN	(32)
void my_read(char filename[32]);

int main (void) {
    setlocale(LC_ALL, "en_US.UTF-8"); //works only with the utf8 file
    my_read("small_latin1.csv");
    my_read("small_utf8.csv");

    setlocale(LC_ALL, "de_DE.iso88591"); //works only with the latin1 and utf8 file
    my_read("small_latin1.csv");
    my_read("small_utf8.csv");

    setlocale(LC_ALL, ""); //in my case (LANG=en_US.UTF-8) works only with the utf8 file
    my_read("small_latin1.csv");
    my_read("small_utf8.csv");

}

void my_read(char filename[32]) {
  char *locale;
  locale = setlocale(LC_ALL, NULL);
  wprintf(L"\nreading file: %s with locale: %s \n", filename, locale);
  wchar_t *p, buf[MAXLINELEN];
  FILE *fp = fopen(filename, "r");

  while (fgetws(buf, MAXLINELEN, fp)) {
    wprintf(buf);
  }
  (void)fclose(fp);
}