Please help parsing file [sed, awk, fortran, bash]

[daniel jimenez]

Hello all,

I need some help fixing the format of some pretty strangely compressed data
files. An example would be like this:

2883
452
0 7
1 6
2
4
6
10 7

Parsing rules:
The first two lines should be ignored.
The first column is the 'index', the second column being the 'counter'.
If there is no second number (ex. index=2), then the second number should
be set to '1'.
If there the index skips (ex. from index=2 to index=4), then the indexes
which where skipped should be set to '0'
Max index is 1024.

That is it. I'd like to be guided to an app (scripting language? awk? sed?
I haven't used those so I really don't know where to start) that can help
me do that effectively.

The command, with the script possibly as the argument, is to be included in
a bash script right before a fortran program is executed as the fortran
program expects the file to be uncompressed and it doesn't seem intuitive
to do it from fortran. Although it would be nice for a guru to let me know
how to handle it from within...

In the end, any solution would be a great help.

Thanks.

-- 
Daniel Jiménez

Re: Please help parsing file [sed, awk, fortran, bash]

[daniel jimenez]

Hello guys,

@richard,

1 this is not homework, a friend asked me to do some statistics on an
experimental dataset he got for his masters thesis and the file was already
that way.

2 i'm well versed in fortran, this problem however is not what i'm used to.
I thought about reading the line bare in character mode, but thought there
should be a more time-efficient (coding time, that is) tool for this type
of problem than some 'tab-finding' code in fortran which would just
obfuscate my code unnecessarily.

3 apart from R code, I'm not used to object oriented oriented programming
and I have only done bash for scripting. I read a bit of the usefulness of
perl, awk and sed a while ago but I haven't had the time to get into that
yet.

4 I thought reading someone's suggestions (and retyping the proposed code)
on this task would be a nice way of learning a few basic concepts on some
of these scripting languages. That is why I stated the problem the way I
did, for others to understand the problem easily and throw some lines of
code at lunch time or something.. I've done that on physics forums for
fortran code or bash scripts (which generally automates fortran program
runs, archival of datafiles and plotting with gnuplot)

5 The data is from a photon counter, I don't know why it is encoded like
that...

@john

I'm having a look at your solution and expect to learn from it. Many thanks!

daniel

Re: Please help parsing file [sed, awk, fortran, bash]

[daniel jimenez]

I didnt mean to say this was oo programming, only that my background is in
strictly scientific computing using structured programming in c and
fortran. this rarely requires doing any formating work on lines such as
this...

as before, the file is from an experimenter who chose the format
arbitrarily (maybe the photon counter outputs that, no clue [dont really
care]) and Im doing this as a favor.

I will be looking into sed as it seems like a really useful tool.

many thanks to all for your inputs

p.s. doing this from fortran would be a chore...

On Fri, Aug 31, 2012 at 7:58 AM, Gaël DONVAL wrote:

> Le vendredi 31 août 2012 à 09:48 -0400, Miles Fidelman a écrit :
> > For going through a file, line-by-line, and massaging the format, my
> > first instinct would be sed.  That's what it's intended for.
>
> Actually, I did not read the whole thread (shame on me) prior to
> answering. I think what is really important here is:
> 1) Where the data are coming from.
> 2) What you want to do with your data.
>
> For instance if it is coming from a fortran program and you have the
> sources of it, you might want to improve the program directly by adding
> a new kind of output to it.
>
> If this is some kind of old files that were generated by a former
> version of the program. Then a quick conversion script using anything
> you want might do the trick: you'll just have to run it during a limited
> time then you will discard it because the newest version of the program
> already provides you with relevant output.
>
> If you are aiming at postprocessing your data, just try to stick to the
> language(s) you choose for that to parse the file.
>
>
> --
> To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact
> listmas...@lists.debian.org
> Archive:
> http://lists.debian.org/1346425113.7366.12.ca...@p76-nom-gd.cnrs-imn.fr
>
>


-- 
Daniel Jiménez

Some Questions with an All Text System

[daniel jimenez]

Hello All,

After some 4 1/2 years of living with a linux-only system and getting
acquainted with several distros, I have finally chosen Debian as my OS.
Ever since I switched about a year ago I have had a wonderful experience
with a community which is strong and helpful.

About 2 months ago I started to test the idea of using a text-only
interface. Previous experience with Guake had made the transition easier
and in a short amount of time I started feeling much more productive when
using the computer. Lynx is my browser of choice since text-mode browsing
seems to counter my extreme tendency to procrastinate and link-surf while
connected to the net. I have added "text"to the kernel line at grub to
prevent running X at boot.

Now, to the questions!

1. How can I configure the system to boot into 'fbterm -S 16' (with the
size option) and then run 'tmux' if I am on a specified virtual terminal.
Say, tty 2 and 3.

2. How can I make it so mc and other programs can use the gpm mouse? If I
set the permissions of /dev/input/mice to "606" It works in the bare tty
term [and works with SDL apps too], but not under fbterm (or tmux under
fbterm).

3. How can I define "mc" as "mc -S gotar.ini"? or "mocp" as "mocp -T
yellow_red_theme" ? Where can I set those aliases (is that it?) at boot?

4. What is a good way to run SuperCollider?

Many thanks to all.

-- 
Daniel Jiménez

framebuffer resolution problem

[daniel jimenez]

Hello everyone,

I've got a problem with my framebuffer (tty) resolution. Currently running
debian testing (through linux mint debian edition).

Grub2 works with my native resolution, which is 1280x800. The rest of the
boot sequence runs at 800x600 or so. After starting X, my desktop runs
hardware accelerated compiz well at native resolution. When I hit
ctrl+alt+f1/f6, the fullscreen terminal looks again really low res.

Can anybody help me?

Many thanks.

-- 
Daniel Jiménez

Re: framebuffer resolution problem

[daniel jimenez]

the requested output is:
$ grep -i buffer /var/log/Xorg.0.log
[24.428] (II) Loading extension DOUBLE-BUFFER
[26.904] (**) NVIDIA(0): Depth 24, (--) framebuffer bpp 32
[29.538] (**) NVIDIA(1): Depth 24, (--) framebuffer bpp 32

upon checking /etc/default/grub

i was able to find
GRUB_GFXMODE=1280x800
GRUB_GFXPAYLOAD=keep

thanks for the help


On Thu, Oct 20, 2011 at 7:01 AM, Camaleón  wrote:

> On Thu, 20 Oct 2011 03:42:59 -0700, daniel jimenez wrote:
>
> > I've got a problem with my framebuffer (tty) resolution. Currently
> > running debian testing (through linux mint debian edition).
> >
> > Grub2 works with my native resolution, which is 1280x800. The rest of
> > the boot sequence runs at 800x600 or so. After starting X, my desktop
> > runs hardware accelerated compiz well at native resolution. When I hit
> > ctrl+alt+f1/f6, the fullscreen terminal looks again really low res.
>
> What does "grep -i buffer /var/log/Xorg.0.log" return?
>
> You can try by editing "/etc/default/grub" file and search for
> "GRUB_CMDLINE_LINUX" and/or "GRUB_GFXPAYLOAD_LINUX" parameters, tweak
> them as required (remember to run "update-grub" after any change).
>
> Greetings,
>
> --
> Camaleón
>
>
> --
> To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact
> listmas...@lists.debian.org
> Archive: http://lists.debian.org/pan.2011.10.20.14.01...@gmail.com
>
>


-- 
Daniel Jiménez



-- 
Daniel Jiménez

Re: framebuffer resolution problem

[daniel jimenez]

> Ah... you're using the nvidia closed driver.

I'm sorry, forgot about the nvidia thing.


> > upon checking /etc/default/grub
> >
> > i was able to find
> > GRUB_GFXMODE=1280x800
> > GRUB_GFXPAYLOAD=keep
> >
> > thanks for the help
>
> If that did it, fine then :-)

Not really.. I found the GRUB_*... but that was already there and my
tty resolution is still the same.

any thoughts?

--
Daniel Jiménez


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/cakuhbgem3hesqs2v-ber6zjfw8tzb0mwr0fwiqaioxkz9cg...@mail.gmail.com

Re: framebuffer resolution problem

[daniel jimenez]

> GRUB_CMDLINE_LINUX="vga=792"
> GRUB_GFXPAYLOAD_LINUX=1280x800x32
>
> After every change remember to run "update-grub".
>
> Greetings,
>
> --
> Camaleón

Thanks! It worked wonders, my ttys work at native resolution now :D

My /etc/default grub has these lines now:

GRUB_GFXMODE=1280x800
GRUB_GFXPAYLOAD=keep
GRUB_GFXPAYLOAD_LINUX=1280x800x32

Hope this helps others too. I'm adding some keywords to this reply.

solution grub nvidia fullscreen resolution tty framebuffer

What a great community this is. Many thanks again.

--
Daniel Jiménez


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/cakuhbggclgnxm2gcfrvcc3zkuntkea7vuyprp_vxhjndoaf...@mail.gmail.com

pgfortran help

[daniel jimenez]

Hello everyone,

I'm currently in the middle of some cuda fortran coding and I really
need a bit of help.

First of all, a brief description: compiling cuda fortran code
(usually *.cuf) requires the use of the proprietary compiler from the
portland group (pgfortran). I've got access to such a compiler on a
computer running  ubuntu 10.04 and recently tried to install it with
some success on Debian after some hacking.

Specifically, I made a couple of symbolic links to help the gnu linker
find crt1.o, crti.o and maybe one more crt*, as the location of said
files wasn't properly set. (something to do with debian-specific gcc
library file paths?)

Now, when I try to compile code using pgfortran, the compiler fails as
it requires "size_t"  to be defined.. and I'm not really well versed
on C or the location of it's libraries. Any ideas that could help?

On a different note, I'd usually used the intel fortran compiler and I
find it *much* faster for regular serial (non-cuda) code. Is it
possible to link two object files produced by different compilers?
Runtime could be much faster by cross-linking.

In any case, I can ssh to work and run everything there ok. So,
getting the compiler running on debian isn't my biggest wish.

Many thanks.

-- 
Daniel Jiménez


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/cakuhbggm6gl0izfydirvgprdk6qbhqm0nxrmtyv+pmyxf+6...@mail.gmail.com

Re: virtualbox just became slow

[daniel jimenez]

I too have the same problem. Although I already had it a couple days
before upgrading to 4.1.4.. upgrading didn't help either.

any thoughts?

Daniel

On Tue, Nov 8, 2011 at 4:54 AM, Steve Kleene  wrote:
> I've been happily running a virtualbox Windows XP machine (VM) on a Wheezy
> host for eight months.  However, the VM just became pathologically slow.  For
> example, if I boot the VM, call Photoshop 6, and open a small JPG, it all
> works but takes several minutes.  During much of this time the XP Task
> Manager pins at 100% CPU usage, and the host's cooling fans are revving hard.
> If I then ask to print the JPG, Photoshop (eventually) declares "There is not
> enough memory for this operation".  When the VM is idle, it shows about 5%
> CPU usage and I see no unexpected memory hogs running.  The Debian host runs
> fine.
>
> This disease roughly coincides with my upgrade from virtualbox 4.1.2 to
> 4.1.4, including the guest additions and extension pack.  So I uninstalled
> 4.1.4 and put back 4.1.2 from snapshot.debian.org, together with its guest
> additions and extension pack.  I also put back a copy of the VM (xpvm.vdi)
> and home vbox directories that I had saved on June 18th when everything was
> fine.  (I saved these as Linux files, not as a snapshot.  I have successfully
> started from them before).  Finally I rebooted the host too.  None of this
> solved the problem.
>
> The host is a  Pentium 4 3.0 GHz with 2 GB of memory.  The VM has the
> recommended 192 MB of memory.  This all worked fine until I ran apt-get
> upgrade (including virtualbox) four days ago.  I have a newer Wheezy machine
> (i5 3.2 GHz + 4 GB) that is running about the same VM (also 192 MB) under
> virtualbox 4.1.4 with no problem.
>
> Any ideas how to identify the source of this slowness?  Thanks.
>
>
> --
> To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
> Archive: http://lists.debian.org/loom.2008t135340-...@post.gmane.org
>
>


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/cakuhbgfh7f+pwhlo6e17_xghs+mfx3evee7sgqdleaoc79_...@mail.gmail.com

rt kernel and nouveau

[daniel jimenez]

Hello everyone,

I'm trying to get an rt kernel working in debian testing on an amd64 laptop
with nvidia graphics.

Ideally I'd have nouveau set up to start when I select (in grub) the rt
kernel and the nvidia drivers when choosing the regular kernel. Problem is,
I don't know how to do that...

Any help appreciated.

-- 
Daniel Jiménez

Re: rt kernel and nouveau

[daniel jimenez]

Wow, thanks a lot guys!

right now, I have two xorg.conf files, backed up like

xorg.conf.nv xorg.conf.rt

the 'nv' one is my old configuration and I use it every time I'm
working/playing/watching a film...

in the other one I removed all references to a second monitor and replaced

driver "nvidia" with
driter  "vesa"

left everything else untouched.


result is: now I just have to switch one file and select the proper kernel
on grub

another quick Q, Is it safe to copy/paste the complete kernel entries on
grub to alter the order they appear?

alternatively, how do you go about changing the default selection on grub2?

I used to know that, but those where grub(1) times :)

Daniel

On Thu, Mar 22, 2012 at 9:28 AM, Sven Joachim  wrote:

> On 2012-03-22 12:59 +0100, Jon Dowland wrote:
>
> > On Wed, Mar 21, 2012 at 11:39:51PM -0700, daniel jimenez wrote:
> >> Hello everyone,
> >>
> >> I'm trying to get an rt kernel working in debian testing on an amd64
> laptop
> >> with nvidia graphics.
> >>
> >> Ideally I'd have nouveau set up to start when I select (in grub) the rt
> >> kernel and the nvidia drivers when choosing the regular kernel. Problem
> is,
> >> I don't know how to do that...
> >>
> >> Any help appreciated.
> >
> > You can prevent the loading of a module on the kernel command line, so
> e.g.
> >
> >   kernel /foo.rt nvidia.blacklist=yes
> >
> > Will prevent the nvidia kernel module from loading.  This will hopefully
> > mean the nouveau one will win the race,
>
> Unless nouveau is blacklisted as well, which is the case if the
> nvidia-kernel-common package is installed.  I would prefer not to
> blacklist anything and boot the non-rt kernel with the 'nomodeset'
> parameter.
>
> > and X will use whatever driver corresponds to what the kernel has
> > loaded.
>
> Not true, nvidia is never autoloaded, so you need at least a 4-line
> xorg.conf.  And it is also necessary to switch the providers of
> /usr/lib/$DEB_HOST_MULTIARCH/libGL.so.1 and
> /usr/lib/xorg/modules/extensions/libglx.so, otherwise OpenGL programs
> will not run.
>
> All doable with a custom initscript, but quite some hassle.
>
> > Getting grub2 to use a different command-line option for each entry left
> > as an exercise for the reader.
>
> I ended up locally diverting /usr/sbin/update-grub (replacing it with a
> symlink to /bin/true) and managing /boot/grub/grub.cfg by hand.
>
> Cheers,
>   Sven
>
>
> --
> To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact
> listmas...@lists.debian.org
> Archive: http://lists.debian.org/878vis38id@turtle.gmx.de
>
>


-- 
Daniel Jiménez

Re: skype?

[daniel jimenez]

It is shameful that, so many years after the digital revolution,
proprietary formats such as those championed by microsoft are *mandated* by
the industry, educational institutions and (in mexico at least) government
departments. In fact, I recently had problems with my yearly tax
declaration due to it only supporting and running on some version of
Internet Explorer.

Hopefully, we can all make that change.

In the case of skype, I understand that their contributions to VoIP have
been pioneering and extremely successful as a tool for individuals and the
enterprise community alike. However, every company that doesn't support
standardization is extremely short sighted and detrimental to the healthy
growth of the field they represent.

Anyway, sorry for the hijack..

Daniel

On Tue, Apr 10, 2012 at 6:04 PM, Christian  wrote:

> On Mon, 09 Apr 2012 18:03:28 -0500, Scott Ferguson <
> scott.ferguson.debian.user@**gmail.com>
> wrote:
>
>
>
>> 
>>
>>
>>
>>> I am concerned with the "big brother" privacy,
>>>
>>
>> Then, sadly, Ekiga is *not* the answer for your needs as it doesn't
>> support encryption. (one of the reasons it's not a fit for my needs).
>>
>> Kphone, Jitsi, Mumble, QuteCom, Linphone, MicroSIP, SFLPhone, and Blink
>> do. Note that not all are multiplatform, see one of my previous posts to
>> this thread for a list of those that are ported to the three main PC
>> platforms.
>>
>>
>> Note also that Skype is encrypted, but Skype has a history of handing
>> out the private key.
>>
>
> The term "handing out" makes me curious. Who exactly are they handing it
> out to?
>
> Personally I am presented with a few requirements from my employer to be
> able to use Linux as my work OS. One is MS Word the other is Skype, with or
> without key handed out.
>
>>
>>
>>
>> 
>>
>>
>> Kind regards
>>
>>
>
> --
> //Christian
>
>
> --
> To UNSUBSCRIBE, email to debian-user-REQUEST@lists.**debian.orgwith a
> subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
> Archive: http://lists.debian.org/op.**wcks80xqp7qh9p@hit-nxdomain.**
> opendns.com
>
>


-- 
Daniel Jiménez

Re: Trouble with Nvidia driver

[daniel jimenez]

I recently had a similar problem with a 9300 gs (which I understand to
be pretty much the same + dx9 support or smth).

Gdm3 would never start after installing a modded kernel. As a
workaround, try changing 'nvidia' for 'vesa' in the grub config file
and see where that leads. Maybe you can boot like that, there wont be
any video acceleration though.

Maybe that kernel version is incompatible with the provided nvidia blob?

Hope it helps.

Daniel

On 4/13/12, Bob Proulx  wrote:
> Eduardo Bearzoti wrote:
>> Video card (lspci output):
>> nvidia Corporation G86[GeForce 8400 GS] (rev a1)
>
> I have a system with the same graphics chip and it is working fine for
> me at 1600x1200 with the default driver.  I am a little embarrassed to
> say that after looking at the /var/log/Xorg.0.log file I am not sure
> which driver it is actually using.  I do not have any special drivers
> (e.g. nouveau or nvidia) installed.  It has been fine and I haven't
> needed to look at it.  It worked out of the box by default.
>
>> when attempting to login trough gdm3, the whole system freezes again.
>
> Since you are reporting system freezes I am concerned that this may be
> a ram problem.  If I were looking at this I would install memtest86+
> as a bootable system.  Then reboot into memtest and let it run through
> several cycles to verify that you are not having a system memory
> problem.
>
>   # apt-get install memtest86+
>   ...
>   Setting up memtest86+ (4.10-1.1) ...
>   Generating grub.cfg ...
>   ...
>   Found memtest86+ image: /memtest86+.bin
>
> That will configure grub automatically for a bootable memtest
> environment.  Reboot and it can be selected.
>
> If I ever find a problem with ram then I am always very thankful
> because replacing ram is pretty inexpensive and a much easier thing to
> do than most other repairs or chasing down driver problems.
>
> Good luck!
>
> Bob
>


-- 
Daniel Jiménez


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/cakuhbggabean1oodr3h6cgsb7i8tdreyfe5hwv6v3rzcgmb...@mail.gmail.com

Re: Is 1600x1200 screen better than 1440x900?

[daniel jimenez]

The extra vertical resolution could come in handy,  although because of the
aspect ratio (its much more of a square than the one you've got now) some
of it wouldn't be useful when watching widescreen movies.

Think about the aspect ratio as one of the most important factors when
choosing a screen. I myself use a 16:10 13" screen at 1200x800 in my laptop
and a 5:4 17" 1280x1024 as a second monitor on my desk.

Daniel

On Tue, Apr 24, 2012 at 11:03 AM, Hans-J. Ullrich wrote:

> Am Dienstag, 24. April 2012 schrieb Sian Mountbatten:
> > When I replaced my desktop with a new computer, I kept my TFT screen. It
> > has the non-standard size of 1440x900.
> >
> > Recently, I have been wondering if it would be better for me to have
> > 1600x1200 screen. Would this be better for Linux software? What about HD
> > TV? And movies?
> >
> > Any comments welcome
>
> TFT should be used in its native resolution. Doing so, you get the sharpest
> screen.
>
> I recommend to use 1440x900 resolution and adjust icon size, fonts etc to
> your
> needs.
>
> Regards
>
> Hans
>
>
> --
> To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact
> listmas...@lists.debian.org
> Archive: http://lists.debian.org/201204242003.44679.hans.ullr...@loop.de
>
>


-- 
Daniel Jiménez