Category Archives: Hacks

Quick and (usually) simple hacks.

Currency Conversion with Javascript Iteration

Missing that handy currency/country conversion link on your favorite page? A quick bookmarklet can fix that.

We will put together some javascript to iterate over our currency elements to do the conversion for us. In this case, we are looking at the bicycle site road.cc and we wish to convert review prices from Pounds to US Dollars.

Screenshot 2013-12-29 22.55.21

road.cc reviews before conversion (http://road.cc/show/review-section/road-bikes/35)

Screenshot 2013-12-29 22.55.31

road.cc reviews after conversion (http://road.cc/show/review-section/road-bikes/35)

 

 

 

 

 

 

 

 

 

We first select all of our data of interest. In this case, our data is in a div with a specific class that merely contains the price.

var data = document.getElementsByClassName("review-price");

Next we will iterate over this data and convert the price using a decent approximation based upon today’s conversion rate. However, we have two considerations; the price div also contains the Pound character which we will first cut out before converting with the replace call and we will also want to round to two decimal places for sanity.

for (var i = 0; i < data.length; i++) {
   data[i].innerHTML = Math.round(data[i].innerText.replace('£', '') * 1.6479 * 100) / 100;
}

If you try and run this, you may notice one caveat – road.cc uses infinite scroll. This means that we will convert material that is currently loaded, however, upon scrolling down and with the loading of additional content, we would have to run the bookmarklet again to convert this newly loaded data. The problem is that we would convert our previously adjusted price data again. To prevent this issue, we can modify our code to convert data that begins with the Pound but not Dollar symbol.

function () {
    var data = document.getElementsByClassName("review-price");
    for (var i = 0; i < data.length; i++) {
        /* change class so we don't suck it up again since we have to re-run the bookmarklet
       again because of infinite scroll
      */
        if (data[i].innerHTML.indexOf("£") != -1) {
            data[i].innerHTML = '$' + Math.round(data[i].innerText.replace('£', '') * 1.6479 * 100) / 100;
        }
    }
}

Now, remember, we wrap this in a javascript declaration to create our bookmarklet. Thus, the final version will result in the following when we place our pieces of code together.

javascript: (

function () {
    var data = document.getElementsByClassName("review-price");
    for (var i = 0; i < data.length; i++) {
        /* change class so we don't suck it up again since we have to re-run the bookmarklet
       again because of infinite scroll
      */
        if (data[i].innerHTML.indexOf("£") != -1) {
            data[i].innerHTML = '$' + Math.round(data[i].innerText.replace('£', '') * 1.6479 * 100) / 100;
        }
    }
}())

Running the bookmarklet should now convert currency data to US Dollars which works with infinite scroll by re-clicking the bookmarklet as new data is loaded.

Accessing Django Template Variables in Javascript

How do we follow the Django DRY principle and avoid as many hard coded elements as possible in our project when we need to include Javascript? A good example of this is if we are pulling in a bookmarklet that will be offered on the site. We need to reference the domain of our project in the bookmarklet, but this then yields code buried in our project’s static files that needs to be altered by anyone that might be utilizing your open sourced code.

This example presents a little bit of a challenge. If this were HTML in our template we could simply reference a DOMAIN variable that we would feed in through a view. However, this is a static Javascript file, so how do we go about utilizing Django template variables in supposedly static Javascript?

Here is a simple method. In our HTML template, we will setup a Javascript global variable that is created to reference a template variable that we feed in from our view, here a domain, and this variable is then referenced in our static Javascript files.

<!-- Our HTML Template (templates/index.html) that is fed data from views.py -->
{% extends "base.html" %}
{% block content %}

<script type="text/javascript" src="/static/somejs.js"></script>

<script> 
var DOMAIN = "{{ site_domain }}";
</script>

<!-- rest of our template, business as usual. -->
{% endblock %}

As you can see, we would feed in a site_domain variable from views.py that might be created something like this.

from django.contrib.sites.models import Site
from django.shortcuts import render_to_response

def index(request):
    '''Our main index view.'''
    site_domain = smart_str(Site.objects.get_current().domain)
    # more of our view code...
    
    return render_to_response('templates/index.html', {
	'site_domain': site_domain,
        #...rest of our template vars
    })

Now, how do we use this in our static Javascript? We reference that global Javascript variable we declared in our template using our Django template variable site_domain.

// Our Javascript file ... /static/somejs.js 
alert("Your current domain is: "+DOMAIN);

Now our code is much more portable, even in places where things should essentially be static.

Setting Default Browser in Linux When Things are Stubborn

Running XFCE4 and setting the preferred applications generally works. However, I noticed that some applications still would not listen to my demands of using a specific browser when launching links. For example, using python’s webbrowser module, and therefore almost any other module opening external links, would use Firefox or Opera (depending on which was available).

# Checking python invocation of a browser
import webbrowser as w
w.get().open('http://mutaku.com')

Here’s the culprit, check in your /etc/alternatives/ directory and you’ll see what I mean.

ls -l /etc/alternatives/

Notice the entry x-www-browser. This will be a symlink to a browser and this value was different than what I was attempting to set through the XFCE4 GUI configuration. This problem seems somewhat common as I’ve seen it occur in KDE and GNOME as well.

To fix this issue, simply run the following command and you should be presented with a list of browsers from which you can set.

# Update your browser choice and choose from the presented text menu
sudo update-alternatives --verbose --config x-www-browser

# Now make sure teh proper symlink is setup
ls -l /etc/alternatives/x-www-browser

Now you should have a functional default choice. Those GUIs can suck it!

Transposing Matrices in Python

Situation:
We have a list of nested lists (list of matrices) that we wish to transpose.

Solution:
Utilize chain to iterate over our master list (x), flattening it, and then we iterate over the elements and return these as a transposed list element.
We can also check that an iterated element exists and meets some value requirement (here, less than 200).

from itertools import chain
[[y[i] for y in list(chain.from_iterable(x)) if y[i] and y[i] < 200] for i in range(0,len(x[0][0]))]

Caveats:
Here, we are assuming that the subunits are invariable in length.

Analyzing Brute Force Attempts in BASH

If you have a public facing SSH server running on the standard port, your message log is probably filled with failed authentication attempts from brute force attacks. Let’s mash up some quick BASH commands to analyze this data. For our purposes, we’ll look at the top attacker IPs and the top usernames tried.

First, pull down all of your message files and decompress them in a working directory using bunzip2.

Once you have all your message logs ready, we will search through them and pull out all of the authentication failure entries and grab the IP and username for each attempt.

grep -r "authentication error" messages* | awk '{split($0,a," "); print a[NF],a[NF-2]}' > attempts

So we first use grep to look for failed authentications by searching recursively for the string “authentication error” in all of our message logs by using the wildcard. We then pipe this to awk and split each input line found into an array delimited by a whitespace. The last part of each line, and therefore our new array, goes something like: ‘authentication error for USERNAME from IP’. So to get the username and IP from our array we can use the array length variable NF and use that to index the variables we need. Here we grab the last using a[NF] and two in from that with a[NF-2]. Finally, we output this to a file called attempts.

Now, let’s use more BASH magic to do the analyses for us. Our attempts file now is in the format as follows: IP USERNAME. We want to see the top IPs and usernames and we can do that with some sorting commands.

cut -d ' ' -f2 attempts | sort | uniq -c | sort -nr > attempts_username
cut -d ' ' -f1 attempts | sort | uniq -c | sort -nr > attempts_ip

Here, we simply grab either the username in the second column or the IP in the first with cut, sort this data, prepend lines with the number of occurrences of each, and then sort by this occurrence number and output to a new file. You can now view attempts_username and attempts_ip to see the top usernames and IPs, respectively, of brute force attacks.

Lastly, we can associate the keep usernames and IPs together and sort on one or the other to see the correlation between the two. To end our initial analyses, we will sort on usernames and find out for the top attempted usernames, what are the top originating IPs.

 sort -k 2,2 attempts| uniq -c | sort -nr | head -n 10

Next time we will be using some GEOIP methods to see where our top attack attempts are originating.

Finding Images with Python

Let’s say we wish to take a directory, the current directory in this example, and gather all the image files. This could be helpful if we were then going to use PIL or Image Magick to manipulate those files, or create a select-able list for use in a GUI, and so on…

There are countless ways we could do this, but here is one with a module that is very helpful with image details.

import os,imghdr
for f in os.listdir(os.getcwd()):
	try:
		if imghdr.what(f):
			print f
	except:
		pass

Bash Snippet for Git Users

Keeping track of what branch you are using while in a git repository can be a bit of a pain. Adding the following code to your ~/.bashrc file can make it a bit easier since it will prepend your prompt with the current branch.

function parse_git_branch {
    GIT_BRANCH=$(git branch --no-color 2> /dev/null | awk '{if ($1 == "*") { printf "(%s) ",$2}}')
    PS1="$GIT_BRANCH\u@\h:\w\$ "
}

PROMPT_COMMAND="parse_git_branch"
Here it is working:
matt@desktop:~$ source /etc/bash.bashrc 
matt@desktop:~$ cd programming/bits
(master) matt@desktop:~/programming/bits$ 

Python – Dynamically Printing to Single Line of Stdout

It’s that time again; code snippet time. This is a short and sweet bit of a code that has some really practical and powerful implications.

When handling large amounts of data processing, it’s often desirable to keep the user informed of where things are in terms of progress. However, filling up the scrollback buffer with thousands of “Completed processing [item X]” or “X % completed” is not always desirable. So, how do we handle verbosity without filling scrollback?

Here, we will print data using stdout but we will continually use the same line of the terminal. It’s extremely simple; before each data print, we will move the cursor back to the beginning of the current line, cut out the entire line of text, and then print our current bit of data.

*Note: This is using ANSI escape sequences and therefore will work on any VT100 type terminal.

 

The code:

import sys

class Printer():
	"""
	Print things to stdout on one line dynamically
	"""

	def __init__(self,data):

		sys.stdout.write("\r\x1b[K"+data.__str__())
		sys.stdout.flush()

 

To output data we might do something like:

totalFiles = len(fileList)
currentFileNum = 1.0

for f in fileList:
	ProcessFile(f)
	currentPercent = currentFileNum/totalFiles*100
	output = "%f of %d completed." % (currentPercent,totalFiles)
	Printer(output)
	currentFileNum += 1

 

And here is an example of what our output would look like in a practical scenario.

Processing some data

Progressing along with dynamic status updates

Simple. You can test it out using an 'for x in range(1,100)' sort of statement as well. Now we can keep a persistent display of data processing or program progress without slamming the scrollback buffer of the user's terminal.

Installing Google Chrome in Slackware 13 (13.37)

Trying to install Google-Chrome via RPM (with rpm2tgz) will give you an error with libnss3. Don’t fret, there is already a way to build Google-Chrome without much effort. Below is a quick way using the mirror from Penn State (carroll), but if you have your install media, you can find the same files there in /extra.

Go to: ftp://carroll.cac.psu.edu/pub/linux/distributions/slackware/slackware-13.37/extra/google-chrome/

The README file is pretty self explanatory, but essentially you want to do the following:

  1. Become rootSetup a build directory (mkdir chrome && cd chrome)
  2. Wget these files: [ ftp://carroll.cac.psu.edu/pub/linux/distributions/slackware/slackware-13.37/extra/google-chrome/google-chrome.SlackBuild ] [ ftp://carroll.cac.psu.edu/pub/linux/distributions/slackware/slackware-13.37/extra/google-chrome/google-chrome-pam-solibs-1.1.3-i486-1.txz ] [ ftp://carroll.cac.psu.edu/pub/linux/distributions/slackware/slackware-13.37/extra/google-chrome/ORBit2-2.14.19-i486-1.txz ] [ ftp://carroll.cac.psu.edu/pub/linux/distributions/slackware/slackware-13.37/extra/google-chrome/GConf-2.32.1-i486-1.txz ]
  3. Change execution bit on slackbuild file and run it (chmod +x google-chrome.SlackBuild && ./google-chrome.SlackBuild)
  4. Go to tmp and install the file (cd /tmp && upgradepkg –install-new google-chrome.txz)
  5. Go back and install the other files (cd /root/chrome/ && upgradepkg –install-new *.txz)

Now you should have a fully functional google-chrome install.