Showing posts with label scripting. Show all posts
Showing posts with label scripting. Show all posts

Wednesday, August 16, 2017

False positive: ansible-lint reports [ANSIBLE0002] Trailing whitespace when there are none

The problem

When running ansible-lint on a Ansible role, it reported [ANSIBLE0002] Trailing whitespace on every line in one of my task files, like shown in the example below:


(vansible23)[audun@hostname my-role]$ ansible-lint .
[ANSIBLE0002] Trailing whitespace
/home/audun/git/my-role/tasks/main.yml:1
---

[ANSIBLE0002] Trailing whitespace
/home/audun/git/my-role/tasks/main.yml:2


[ANSIBLE0002] Trailing whitespace
/home/audun/git/my-role/tasks/main.yml:3
- debug: msg="Hello World"



The investigation

I had a suspicion there might be hidden characters in the file, but before running it through a HEX editor, I tried to check it with the Linux file command:


(vansible23)[audun@hostname my-role]$ file /home/audun/git/my-role/tasks/main.yml
/home/audun/git/my-role/tasks/main.yml: ASCII text, with CRLF line terminators



As shown above, it turned out that the file had CRLF line terminators on every line, indicating that this file had been created on Windows. That was luckily easy to fix.


The solution

The dos2unix command was used to change the file to Unix format:


(vansible23)[audun@hostname my-role]$ dos2unix /home/audun/git/my-role/tasks/main.yml
dos2unix: converting file /home/audun/git/my-role/tasks/main.yml to Unix format ...



Afterwards ansible-lint reported no issues:


(vansible23)[audun@hostname my-role]$ ansible-lint .

Wednesday, November 9, 2016

How to install Vagrant with Oracle® VM VirtualBox and libvirt support on Antergos Linux

Installation

I recently needed to install Vagrant with support for Oracle® VM VirtualBox and libvirt for my dixie project at GitHub. Some of the virtualbox dependencies was found in this excellent post over at forum.antergos.com. Then I wrapped it into a script, and the here is the part of the script that installs and configures it:

if [ $(uname -r | grep ARCH | wc -l) -gt 0 ]; then
  sudo pacman -S --needed --noconfirm vagrant
  sudo pacman -S --needed --noconfirm libvirt
  sudo pacman -S --needed --noconfirm linux-headers
  sudo pacman -S --needed --noconfirm virtualbox virtualbox-guest-iso
  sudo pacman -S --needed --noconfirm vde2 net-tools virtualbox-ext-vnc virtualbox-host-modules-arch
  sudo pacman -S --needed --noconfirm ansible rsync
  sudo su -c "modprobe vboxdrv" || echo "Reboot your computer and try again"
  if [ ! -f "/etc/modules-load.d/virtualbox.conf" ]; then
    sudo touch /etc/modules-load.d/virtualbox.conf
  fi

  if [ $(grep vboxdrv /etc/modules-load.d/virtualbox.conf | wc -l) -eq 0 ]; then
    sudo su -c 'echo "vboxdrv" >> /etc/modules-load.d/virtualbox.conf'
  fi

  if [ $(grep vboxnetadp /etc/modules-load.d/virtualbox.conf | wc -l) -eq 0 ]; then
    sudo su -c 'echo "vboxnetadp" >> /etc/modules-load.d/virtualbox.conf'
  fi
  if [ $(grep vboxnetflt /etc/modules-load.d/virtualbox.conf | wc -l) -eq 0 ]; then
    sudo su -c 'echo "vboxnetflt" >> /etc/modules-load.d/virtualbox.conf'
  fi
  if [ $(grep vboxpci /etc/modules-load.d/virtualbox.conf | wc -l) -eq 0 ]; then
    sudo su -c 'echo "vboxpci" >> /etc/modules-load.d/virtualbox.conf'
  fi
fi

Install Ruby with gem support on Red Hat® Enterprise Linux

This post is heavily influenced by http://collectiveidea.com/blog/archives/2011/10/31/install-ruby-193-with-libyaml-on-centos/, so all credit to that author.

The problem

Installing a Ruby gem with gem install on  Red Hat® Enterprise Linux 6.8 returns errors indicating problems with libyaml:

gem install bundler
/usr/local/lib/ruby/1.9.1/yaml.rb:84:in `<top (required)>':
It seems your ruby installation is missing psych (for YAML output).
To eliminate this warning, please install libyaml and reinstall your ruby.

The investigation

Trying to reinstall libyaml and libyaml-devel followed by a reinstall of ruby did not solve the issue.

The solution

After investigating this issue for a while, and not able to solve it through yum, I considered to either install and use rvm for this, or to download and compile the source. I found  http://collectiveidea.com/blog/archives/2011/10/31/install-ruby-193-with-libyaml-on-centos/ where the author had good success with the latter, so I did the same, but cleaning up all by Ruby and libyaml stuff from yum first:

yum remove -y ruby libyaml libyaml-devel rubygem-rake rubygems rubygems-devel

# Download and install libyaml
cd /tmp
wget http://pyyaml.org/download/libyaml/yaml-0.1.7.tar.gz
tar xzvf yaml-0.1.7.tar.gz
cd yaml-0.1.7
./configure --prefix=/usr/local
make
make install

# Download and install Ruby
cd /tmp
wget http://ftp.ruby-lang.org/pub/ruby/2.3/ruby-2.3.1.tar.gz
tar xzvf ruby-2.3.1.tar.gz
cd ruby-2.3.1
./configure --prefix=/usr/local --enable-shared --disable-install-doc --with-opt-dir=/usr/local/lib
make install

# Install your gems
gem install bundler

Wednesday, June 29, 2016

The art of creating efficient algorithms

How often do we think outside the box to create a creative and efficient algorithm when we develop software? I bet answer is "probably not often enough". I have not been coding in Python for close to two years. In fact I have not been coding much for the past two years in any language apart from a few Java utilities and the odd shell scripts to help me get things done in a predicable and automated manner. Hence this is not a post about how to create good algorithms, but rather some reflections on code performance during my re-entry to Python programming. So please forgive me for any false assumptions and poor code quality.

While reading a book on Python 3 recently, I came across an interesting anecdote about the German mathematician Carl Friedrich Gauss, who at the age of 8 was given a homework assignment to sum up all the numbers between 1 and 100. To his teacher’s astonishment Carl Friedrich answered 5050 a few seconds later, and it was the teacher had to go home and check if the answer was correct.

Instead of adding every number between 1 and 100, the young Carl Friedrich quickly realized that there were 50 pairs of unequal numbers that would sum up to 100:
99 + 1
98 + 2
And so on, all the way down to 51 + 49
And since he had not included the number 50 yet, the answer was (50 pairs * 100) + 50 = 5050

So the youngster had by himself discovered a pretty efficient algorithm for summing up integers. Not bad for an 8 year old! According to Wikipedia this story is contested, but nevertheless it is a very good example of a creative solution for optimizing an algorithm.

Naturally I needed to test the same logic on a computer, so I wrote a small Python script to check the human way of thinking (the count()method) and the young Carl Friedrich' way of thinking (the f_count() method):

import time

class FunnyMath(object):

    def __init__(self,num=0):
        self.num = num
   
    def count(self,num=0):
        start = time.time()
        self.num = num;
        for i in range(1,num):
            self.num += i
        end = time.time()
        print("Traditional approach took {} seconds to get the result: {}".format(round((end-start),2),int(self.num)))
    
    def f_count(self,num=0):
        start = time.time()
        self.num = num**2/2.0 + num/2.0
        end = time.time()
        print("Creative approach took {} seconds to get the result: {}".format(round((end-start),2),int(self.num)))
    
def main():
    fm = FunnyMath()
    num = 100000000
    fm.f_count(num)
    fm.count(num)
   
if __name__ == "__main__":
    main()

How long it takes to run will of course depend on your hardware resources. I tested this script on two different laptops.

On the "fast" laptop I got the following output:
Creative approach took 0.0 seconds to get the result: 5000000050000000
Traditional approach took 14.88 seconds to get the result: 5000000050000000

On the "slow" laptop I got the following output:
Creative approach took 0.0 seconds to get the result: 5000000050000000
Traditional approach took 27.76 seconds to get the result: 5000000050000000

So it is an interesting example that shows that some algorithms can and should be optimized. Not only did f_count() perform better. It also had fewer lines of code.

 But with a rewrite I could also optimize the slow count() method:

import time

class FunnyMath(object):

    def __init__(self,num=0):
        assert type(num) is int, "num was not an integer {}".format(num)
        self.num = num
   
    def count(self):
        num = self.num
        for i in range(1,num):
            num += i
        return int(num)
        
    def f_count(self):
        return int(self.num**2/2.0 + self.num/2.0)
    
def main():
    fm = FunnyMath(num = 100000000)
    start = time.time() 
    num = fm.f_count()
    end = time.time()
    print("Creative approach took {} seconds to get the result: {}".format(round((end-start),2),num))
    
    start = time.time()
    num = fm.count()
    end = time.time()
    print("Traditional approach took {} seconds to get the result: {}".format(round((end-start),2),num))
    
if __name__ == "__main__":
    main()

Then I ran it on the "fast" laptop and got the following output:
Creative approach took 0.0 seconds to get the result: 5000000050000000
Traditional approach took 5.98 seconds to get the result: 5000000050000000

A full rewrite was not really necessary, but I wanted a cleaner and more compact code for my class. The observant reader might have spotted the real problem with the first definition of the count() method:

def count(self,num=0):
        start = time.time()
        self.num = num;
        for i in range(1,num):
            self.num += i
        end = time.time()
        print("Traditional approach took {} seconds to get the result: {}".format(round((end-start),2),int(self.num)))
 
For each iteration in the loop I was updating the class variable num instead of just using a local variable called num, so the same performance gain for the count() method could have been achieved with a small rewrite:

def count(self,num=0):
        start = time.time()
        num = num;
        for i in range(1,num):
            num += i
        end = time.time()
        print("Traditional approach took {} seconds to get the result: {}".format(round((end-start),2),int(num)))

When I ran that on the "fast" laptop I got the the following output:
Traditional approach took 5.92 seconds to get the result: 5000000050000000

The reason for this performance gain is explained in the Python wiki.

Still, the count() method would always be slower than the f_count() method.