Showing posts with label linux. Show all posts
Showing posts with label linux. Show all posts

Saturday, July 28, 2012

Working With Binary Data in Python

This puzzled me for quite a time, so here is a post explaining some of my findings.
First we define a bytes array:
 s = bytearray(b"Hello World")
 for i in s: 
     print i
 ...
 72
 101
 108
 108
 111
 32
 87
 111
 114
 108
 100

now, let's write this data to a file:
 f = open('helloword.bin','wb')
 for i in s:                                                                                                                                       
    f.write(struct.pack("I",i))
 f.close()

Let's inspect the file created:
 
 $ du -h points.bin
 4.0K    points.bin
 $ file points.bin
 points.bin: data
 $ less points.bin
 "points.bin" may be a binary file.  See it anyway?

Binary file sizes

let's us write "hello world" into a text file in a text form:

 f = open('helloword.txt")
 f.write("hello world")
 f.close()

Once again we can inspect the file:

$ du -h   helloworld.txt
 4.0K    helloworld.txt


Now, what happens if we make a longer binary array?


 s.split()
 s.append(33)
 bytearray(b'Hello World!')
 for i in range(10000): s.append(33)
 f = open('longhelloword.bin', 'wb')
 import struct      
 for i in s: f.write(struct.pack("I",i))
 f.close()
 f = open('longhelloword.txt', 'w')
 hello = "Hello World!"
 for i in range(10000): hello = hello+"!"
In a shell, examaining the file sizes:
 
 $ du -h longhelloword.bin
 40K     longhelloword.bin
 $ du -h longhelloword.txt
 12K     longhelloword.txt

Wait a minute ! Why is the binary file almost 4 times bigger?

The answer is: it depends on the format specifier in struct.pack.
Namely, we used an unsigned int, for each character we then reserved 4 bytes!.
When we saved the text, every character was assigned to the file exactly as a char which takes one byte only.

If we repeat the above with struct.pack("b",i) the sizes of the file won't differ:

 f = open('longhellowordwithchar.bin', 'w')
 for i in s: f.write(struct.pack('b',i))
 f.close()

and in the shell:

 $ du  longhellowordwithchar.bin
 12      longhellowordwithchar.bin
 $ du  longhelloword.txt       
 12      longhelloword.txt

Credits:
http://dabeaz.blogspot.de/2010/01/few-useful-bytearray-tricks.html
http://docs.python.org/library/struct.html#module-struct

Saturday, December 10, 2011

The Wierdest Post Ever in ServerFault.

I don't know if this is a joke or not. But if it's not a joke, it must be cruel. Just take a look on that guy's question and look at the answers.
This is probably the worst system administration task.

Wednesday, October 26, 2011

Analyst my S

Sometimes I just can't wonder ... An idiot posts a blog which shows the guy has no clue what he's talking about. And people actually make such a fuss about it.

The guy writes here:

Sure, Android is built on top of Linux, but Linux is only one of many piece parts of the Android mobile operating system. It is not Linux.

Anyone who knows what a Linux distribution is, would agree that no Linux distribution is Linux according to this definition. 

So, Mr. Analyst my S. Next time I meet people with your title, I will seriously think how I should tag them. I would even bother suggesting this guy takes a "Non-Linux" like Debian or Ubuntu for a try. Idiots like this guy deserve to punished for the rest of their lives sitting in front of a computer with Windows 7. 

Wednesday, October 19, 2011

When friends spit FUD

Warning: This is not a technical post, rather just a boring rant and a call for advice. A friend of mine asked me what programming language he should learn. He was thinking of dwelling into modeling after doing some programming with R and Matlab. Then, I suggested he should give Python a spin. I think python has much nicer syntax than R or Matlab. Regarding speed, I have seen many many times that Python is simply faster than Matlab and R when using modules written in C or Fortran (which is the case for Numpy and most of scipy). And finally, both R and Python are FREE, so their amount of packages is extensive so every user has a great choice of ready made recipes. So, it seemed my friend was kind of convinced and thought of trying Python. But a couple of days later, when we chatted, he told me the following sentence:  

"2. anything that's open source is inherently less user friendly than commercial products."

Now, I am a dickhead, so I get mad when friends of mine, whom I consider to be smart, say stuff like that. Especially, when they sound so convinced. If you read my blog, you are likely to label this sentence as FUD. And you are likely to disagree like me. Personally, I don't know how to change a man's opinion who is so convinced. I also don't know if I should try. But it sure does piss me off and sets me on angry mode. It would be nice to know what other people do in such cases. I just don't want to be an evangelist anymore.

So, please give me and advice how to fight FUD when it comes from friends you really love.

Finally, apologies to my friend who got quoted without reference, and had to experience me getting upset.

Thursday, October 06, 2011

Another crack in the Windows hagmony

For about 4 years now there is a possibility to get a Netbook with Linux. Now, there are many of them coming with Android and Meego. Asus has been shipping Linux for quite some times. Unfortunately, always as a second citizen in it's laptops. The better models always come with Windows. IMHO, it's stupid, because WIN7 just kills the system - i.e. WIN7 Boots slower than WINXP although the hardware today is MUCH faster. But back to what I wanted to write about. Today this really caught my eyes browsing some laptops:

A Windows agnostic "WIN" button. I've seen that already in laptops which come with Ubuntu pre-installed, but first time I see it with a main line manufacturer! Such a shame that the same model which comes in white does have the Win button. The laptop by the way is ASUS Eee PC X101-BLK022G which comes pre-installed with Intel's Meego Linux distribution.

Monday, October 03, 2011

Quicktip: make gedit more sane

Continuing the agile setup of GNOME desktop using gconftool-2. Here's a snippet to make tab width 4 spaces, insert spaces instead of tabs and enable automatic indentation:
gconftool-2  --set /apps/gedit-2/preferences/editor/tabs/insert_spaces --type bool 1
gconftool-2  --set /apps/gedit-2/preferences/editor/tabs/tabs_size --type int 4
gconftool-2  --set /apps/gedit-2/preferences/editor/auto_indent/auto_indent --type bool 1
Well, GNOME 2 isn't that evil. Let's hope the Mate Desktop Environment really catches.

Monday, September 26, 2011

Linux users: stop whining about UEFI Secure booting!

Sorry for the provocative title, but I think Linux and other FOSS Operating System should really stop whining about the UEFI secure booting.
I see no difference between someone who buys a laptop made by Apple and a someone who buys a Laptop with Windows OS. As the saying goes: "If you sleep with dogs, you wake up with flees". For a very long time the Linux Community ignored the WINTEL monopoly and did very little to support free hardware manufacturers. Many people would invest tons of time in hacking and patching Linux and drivers where they where manufacturers ignored deliberately or unintentionally Linux.
I don't buy hardware very often. And so most people who use Linux don't need to. But when I do buy hardware or asked to recommend hardware, I never do buy anything with Windows, or NVIDIA, or ATI.
I won't even answer friends or family question in the style "can you recommend this WinLaptop or that WinLaptop?". Sounds extreme, I know. But in fact, my strategy works. I have successfully bought my own Laptop from a small company that sells Clevo laptops with out windows. And more than 5 friends of mine who saw "Linux Just Works" out of the box, decided to buy these laptops too. I think it shows that there is a Market for people who are not interested in Windows. Even if it is small, it is growing.
But besides looking for a Laptop which does not come with Windows. You can specifically buy Hardware from manufacturers that do support Linux. Nowadays the ARM architecture is  rising, and provides computational power which is enough for most home users. In fact, the beagleboard provided more computational power than my 6 years old Laptop which still works. Not only that, the architecture is completely open, so I know, no UEFI Secure boot will shit on my head from the manufacturer.
There are a few more hardware manufacturers which make nice and OPEN hardware. To mention a few:
gumstix, opencircuits, hawkboard, pandaboard, igep and even more.
And there is even a whole Laptop/Tablet based on beagle board. 
You might mock the  "lousy" or humble hardware offering in this free field. But in fact it is enough for text processing and multimedia. It is enough for me, and I know, my next laptop, will be based on ARM, because I give a #!$% about UEFI, and I don't care what Microsoft does. Instead of fighting it directly, I will give my money to companies which promote my computational freedom.
I hope this post will encourage at least one extra person to re-consider his next WINTEL/AMD/NVIDIA buying.


Saturday, September 10, 2011

How to: Bluetooth Speakers without Pulseaudio

About 3 months ago I have purchased a Bluetooth Speakers set. I chose the Creative Labs D200, which provide me with nice tunes compared to the prices. The speakers worked out of the box with my Bluetooth adapter. Pulseaudio also makes life easier to switch between the laptop's own speaker to the Bluetooth speakers. However, I get irritated sometimes with pulseaudio, and I was looking for an alternative solution.

After researching a few different sources I have come up with my own flavor of solution, which I document here for future benefit of myself and others.

Thursday, August 25, 2011

20 Years of linux - a humoristic view

From an Israeli Linux forum:

<KingYes> Today, 20 years ago, Linus announced he is working on a new operating system
<KingYes> http://groups.google.com/group/comp.os.minix/browse_thread/thread/76536d1fb451ac60/b813d52cbc5a044b?pli=1
<KingYes> This is Great, how time passes by.
<Haimn> WOW I didn't know his post still exists.
<Guest> In windows it would not happen...
<KingYes> What would not happen?
<Guest> Your lack of sense of humor.


Credit: Original Link in Hebrew


I found it amusing, so I think it was worth a translation. I tried submitting to bash.org, but it does not work ...

Tuesday, July 05, 2011

GNOME Fork revisited

Since I posted in my blog about GNOME 2 fork, I have seen numerous hits. In 4 months since than,
this single post had more than 3 times hit compared to my second most popular post about TMUX copy mode.
Yesterday, I bumped into a young fork of GNOME 2, called matte. Named after the Argentinian herb infusion. Let's hope this project will not have a bitter end like the drink, and rather long sweet life. The odd choices of developers are sometimes annoying. While it might seems narrow minded of me, I do expect a project that aims to be the size of GNOME2, to write a document or two in English. He, but what do I know there are actually more Spanish speaking people. I should learn Spanish instead of blogging...
I will try Mate sometimes probably.But in the mean while, thanks to GNOME developers I have been looking into alternatives and found my own peace within DWM, and now I am building my Desktop from Scratch.


Tuesday, June 21, 2011

The best latex editor Kile 2.1 Released!

This should be really bigger news! Although I don't use KDE at all, Kile is the best editor for Latex, I have tried. The developers released 5 beta versions before, which never gave me any problem.
11 Days ago, kile 2.1 was finally released, here is the release announcement .
I love how simple Kile is, although it is full with goodies and options. I love how it is out of the way when you want it. I tried contributing to Gummi-latex editor. But eventually, I found it useless.
I love the VIM mode in Kile, and I wish it was possible in some GTK+ Editor. Yeah, I know I can use vim-latex-suite. But I am too lazy to learn it :-). So, I use vim for all other programming, and Kile for Latex.
So, bottom line, I really recommend to download and install latest Kile!

Sunday, May 29, 2011

The future of Windows

Last week I participated in a Windows 7 course. I was sent to the course by my employer, although in my official role is Linux System Engineer. 9 people attended the course. Looking around my colleagues I had some interesting observations, I'd like to share.
First of all, among the participants, I was the youngest. 2 Other guys where also Linux admins, the rest were Windows admins. Some external appearances really struck me. Usually, Linux guys have the image of being wild hairy beasts with long unshaven beards. Well ... there are some differences with the Windows guys, but not what I expected:
  • All the Linux guys were younger then 35, windows guys are older. This is a very strong indicator about the future of this OS.
  • The Linux Guys (including me) dress sportive casual stuff, short pants, some sport brand cool shirt or just plain T's. Windows guys are dressed business casual: jeans or some fancy pants, with a button up shirt tucked in their pants behind some leather belt usually holding a role over fat stomach.
  • Linux Guys come to the course cycling (MTB or Road bike), Windows guys all came with a Car, which can explain the fat stomach phenomena.
  • Out of 6 guys, 3 guy actually had facial hairs, and our instructor a mustache! Personally, I find people with mustaches unreliable.
This are just my few observations on being in a Windows 7 course.

Another anecdote I have noticed while learning Windows 7, is it's lousy backward compatibility. Windows 7 does not include Solitaire or Minesweeper, at least in the VM's we had ... Imagine, passing 2 days in front of the most boring OS, without minesweeper...


Thursday, March 31, 2011

running more than one linux distribution without rebooting ...

My laptop has reached Nirvana with Debian Squeeze, everything works out of the box, and I very happy with the stability of the system. I don't feel the software is outdated, because I anyway install some applications I need or like directly from source.
However, I still want to be able to run Debian testing, and see how my system evolves. Enter "chroot".

With chroot I can run more that one Linux Distro and more than one X-Server with out the need to stop my work and reboot my laptop. This is just great.

Here is how to do it, assuming you already have a partition with a Debian installed on in do the following (if you don't have a partition with Debian installed, I suggest you take a look at "debootstrap"):

1. First make a location where you will work as the root of the new debian, in my case:
$ su -
# mkdir /debian-sec
2. now mount that partition, in my case /dev/sda5:
mount /dev/sda5 /debian-sec
3. enable important devices so the new Debian will work as expected.
mount -o bind /proc /debian-sec/proc
mount -o bind /dev/ /debian-sec/dev/ 
mount /dev/pts /debian-sec/dev/pts -t devpts 
mount -t sysfs /sysfs/ /debian-sec/sys 
4. finally change the root with chroot:
chroot /debian-sec/ /bin/bash
If you want to run a secondary X-Server, do the following inside the chroot:
chroot # vi /etc/gdm/gdm.conf # do s/vt7/vt9/ in [servers] section
chroot # /etc/init.d/gdm start
this will start another X-Server on vt9, to which you can switch with Alt+Ctrl+F9.

Cheerios, Linux and Debian !

UPDATE: The last section about GDM in chroot seems not to work in gdm3...
with gdm3:

chroot # vim /usr/share/gdm/gdm.schemas

change the following keys:
   <schema>
      <key>daemon/FirstVT</key>
      <signature>i</signature>
      <default>9</default>
    </schema>



What does work ?

Switch to tty1, login as root, and chroot into the chrooted debian.
Then chanage to with "su - "

now start the X server as the following:
startx /usr/bin/fluxbox -- :1&
or
startx /usr/bin/gnome-session -- :1&
or
startx  -- :1&


Friday, February 18, 2011

Wow, Long time ago ... My last post ages ago was a rant about Uhbuntu ...
Needless to say that since then, I have quit using Uhbunutu as my main Desktop, and moved to Debian and ArchLinux. I also play extensively with OpenBSD and FreeBSD, and at work my workstation is OpenSUSE from which I maintain mostly CentOS servers. But no Uhbuntu ...

Tuesday, November 06, 2007

Ubuntu Gutsy is Bloatware. The latest ubuntu distro is terrible, comparing with the previous one, Feisty Fawn.
Cases like this make me think about the 6 months release cycle, and really appreciate my Debian stable, which is old, but working. So why am I saying Ubuntu Gutsy is terrible ?
First, upgrading breaks USB mounting. During Beta testing phase many people reported that upgrading will break mounting of external disks. Well, the guys in Canonical think this is not important, and released without supplying an adequate solution.
This is wrong, because it hearts product loyalty. Many users, like me are loyal until something just happens. I've used Ubuntu from version 6.04 to 7.10 as main laptop working station. Prior to this, I've use SuSE, until their terrible package management drove me away ! Now with Ubuntu 7.10 being so bad, I looking for alternatives.
Second, Gutsy has many great features, but do we need them all ? Canonical, has done a great job winning the linux desktop, and now is aiming for the server and corporate market. But I suspect that some decisions are just to biased. For example, the inclusion of AppArmor and SELinux. Security is important feature of any OS and linux is already doing a great job. Do desktop users need such enhanced security as AppArmor or SElinux, which is implemented in the NSA ?
Well, the reason I rant about that is that, both services slow down boot times significantly, and remind me the days I've been working with OpenSuSE 10.1, which was slugish. I used ubuntu 7.04 on a small old laptop, which has a 500 MHz processor and 128 MB of RAM. Boot time from GRUB to GDM took 1:47 minutes. With the new kernel and boot process it takes over 5 minutes!
Now, advanced users can of course disable AppArmor on boot time, but try removing the package libselinux - everything is dependent on it ! So it won't be removed.
So, old laptops will just work terribly slow with gutsy ! So does Tracker slows things down, and behaves in awkward ways. Tracker, SElinux, and AppArmor, are all great features, but are terribly heavy, and should not be installed on old computers. Hardware detection should do better and at least ask the user if they should be activated by default.
Eventually, I've ended up, burning another feisty CD, and reinstalling it. I'll wait and see what will be with ubuntu 8.04. If it won't work to my liking I'll completely switch to Debian. My server runs on it and I'm happy with it.