Using SHA-1 to generate passwords

Posted on 2012-03-10 by Johan

This is probably the worst thing about the Internet, remembering passwords.

Until about a year ago I had the same password for almost every service, sometimes I spiced it up by adding a character in the start representing the service (F for facebook, G for Gmail and so on).

A colleague had a different approach, he used simple “passwords” which he then computed SHA-1 on. He had developed a simple java script that performed the SHA-1 conversion.

I have changed his original script a bit to be able to generate different length passwords (11, 21 and 26 characters).

You can try it out yourself, just add the following url as a bookmark (works in all browsers, I think):

javascript:(function()%7Bvar%20s%20=%20document.createElement(%22script%22);%20s.src%20=%20%22https://superfredag.com/convert.js%22;%20void(document.body.appendChild(s));%7D)()

To use it, put a “-” in the password box and press the link, this will give you a glass pane with a password box, just write your simple password (Facebook1234 for example) and choose the length, the script will enter the password in the field where you put the “-”.
If you choose Facebook1234 as your simple password the 21 character SHA-1 will be fc1a17377c7ed19872037.

The convert.js script is YUI-compressed, the original one is convert.js.original.
There is also a form based version if you just want the password in cleartext:

https://superfredag.com/convert.html

Of course everything is on GitHub if you want to look at it and put it up for yourself. If you don’t have any web hosted you could just put the scripts in the Public part of your Dropbox.

Polygon with holes in aggdraw

Posted on 2012-03-06 by Johan

Drawing polygons with holes have haunted me for a while and recently I had to do it in Python with PIL.

Googling the subject gives you a hint.

This code:

import os
import Image
import aggdraw

draw = aggdraw.Draw('RGB', (100, 100), 'white')

path = aggdraw.Path()
path.moveto(10, 10)
path.lineto(10,60,60,60)
path.lineto(60,10)
path.lineto(10,10)
path.moveto(20,20)
path.lineto(40,20)
path.lineto(40,40)
path.lineto(20,40)
path.lineto(20,20)

pen = aggdraw.Brush("black")
draw.path((25, 25), path, pen, None)

img = Image.fromstring('RGB', (100, 100), draw.tostring())
p = os.path.join(os.path.dirname(__file__), 'box.png')
img.save(p)

Gives the following image:

Compiling scipy in 32 bit in a 64 bit environment (el5)

Posted on 2012-03-01 by Johan

During the last 2 days I’ve been trying to compile an old product in 32 bit mode on a 64 bit Redhat Enterprise Linux 5 environment which should not be that hard.

Python itself is no problem:

TCC="gcc -m32" ./configure

(got information from here)

And this approach works for almost every 3rd party software, except for Scipy.

Scipy contains a lot of FORTRAN code and it wasn’t obvious how to get setup.py to understand that it should both build and link with the -m32 flag.

After a lot of trial and error this is what I used:

F90FLAGS="-m32" F77FLAGS="-m32" \
LDFLAGS="-g -Wall -shared -m32 -fPIC" \
$PYTHON setup.py config_fc --fcompiler=gnu95 install

There is probably other flags that are better but these worked for me, I think the LDFLAGS is what did it since they are used when g77 is linking the FORTRAN code. The flags also worked fine for numpy.

I will try to write more often but I have had a lot to do recently.

Using your smartphone in the US on vacation (T-mobile rocks)

Posted on 2011-10-09 by Johan

This is something that has been discussed on forums everywhere and there is no real answer.

Just arrived in San Francisco with my swedish iPhone 4, went into a T-mobile store and bought a Monthly4G package for 50$, this gives me unlimited data, talk and text for one month.

Added 10$ and I got free text and landline calls to Sweden.

The only bad thing is that T-mobile 3G network uses a frequency that the iPhone (and many smartphones) cannot use so I’m limited to Edge data transfer.

But still this is very cheap and if you travel with someone you can call them for free while on vacation.

Login on Microsoft IAS from IOS SDK (cookie/form based login)

Posted on 2011-09-27 by Johan

Recently I had to authenticate from an iOS device against a Microsoft IAS. Had never done this before and it was not that easy to find out how so I’m posting my solution here.

For the record, I am not sure which version of IAS I’m authenticating against and actually I’m not even sure it’s IAS but still this solution should work for form based authentications.

I’m using ASIHTTPRequest in iOS.

- (void)loginWith:(NSString*)userName andPassword:(NSString*)password {
    NSLog(@"Login with user: %@ and pass: %@", userName, password);
    NSURL *url = [NSURL URLWithString:
        @"https://domain.com/CookieAuth.dll?Logon"];
    ASIFormDataRequest *request = [ASIFormDataRequest requestWithURL:url];
    [request setPostValue:@"Z2F" forKey:@"curl"];
    [request setPostValue:@"0" forKey:@"flags"];
    [request setPostValue:@"0" forKey:@"forcedownlevel"];
    [request setPostValue:@"20" forKey:@"formdir"];
    [request setPostValue:@"4" forKey:@"trusted"];
    [request setPostValue:userName forKey:@"username"];
    [request setPostValue:password forKey:@"password"];
    [request setPostValue:@"Log On" forKey:@"SubmitCreds"];

    [request setDelegate:self];
    [request startAsynchronous];
}

- (void)requestFinished:(ASIHTTPRequest *)request
{
    if (request.responseStatusCode == 400) {
        NSLog(@"Invalid code");
    } else if (request.responseStatusCode == 403) {
        NSLog(@"Code already used");
    } else if (request.responseStatusCode == 200) {
        NSString *responseString = [request responseString];
        NSLog(@"Response %@", responseString);
    } else {
        NSLog(@"Unexpected error");
        NSLog(@"%@", [request responseString]);
    }
}

ASIHTTPRequest takes care of the cookie so it will automatically be passed to the server on requests.

This is really just a plain form login and really has nothing to do with Microsoft IAS.

Cloud Computing Economies of Scale

Posted on 2011-08-30 by Johan

Just watched a very interesting session on Cloud Computing by James Hamilton from MIX10.

http://channel9.msdn.com/events/MIX/MIX10/EX01

Really interesting about why you should use cloud computing instead of buying servers. Personally I already run my personal stuff in EC2, S3 and GAE and I will not go back to running an SMTP-server in the closet.

Also my ISP had a 10 day outage this summer and emails don’t like that

Google Maps polygons with holes

Posted on 2011-08-12 by Johan

The last couple of weeks I have been experimenting with Google Maps trying to draw filled polygons that look alright. I’m using matplotlib for making the polygons and I’ve figured out that the output from contourf(…) is like a plotting routine where you first get a polygon that should be filled with the current level and the following ones are holes in it.

Before I just draw them all, coloring the holes with a lower color. This forced me to sort the polygons according to size which worked ok but didn’t look good.

In Google Maps v2 there was something called encoded polygons but it seems as if they were removed in v3 (never supported by Chrome anyway).

Anyway, the correct way in v3 is to do like this:

  var paths = [[
    new google.maps.LatLng(38.872886, -77.054720),
    new google.maps.LatLng(38.872602, -77.058046),
    new google.maps.LatLng(38.870080, -77.058604),
    new google.maps.LatLng(38.868894, -77.055664),
    new google.maps.LatLng(38.870598, -77.053346)
  ], [
    new google.maps.LatLng(38.871684, -77.056780),
    new google.maps.LatLng(38.871867, -77.055449),
    new google.maps.LatLng(38.870915, -77.054891),
    new google.maps.LatLng(38.870113, -77.055836),
    new google.maps.LatLng(38.870581, -77.057037)
  ]];

  function initialize() {
    var map = new google.maps.Map(document.getElementById("map"), {
      zoom: 16,
      center: new google.maps.LatLng(38.8714, -77.0556),
      mapTypeId: google.maps.MapTypeId.SATELLITE
    });

    var poly = new google.maps.Polygon({
      paths: paths,
      strokeWeight: 3,
      fillColor: '#55FF55',
      fillOpacity: 0.5
    });

    poly.setMap(map);
  }

My project is now online at http://halvklart.se/ and here’s a screenshot:

There are some troubles with the polygons when applying the b-splines but I will try to take care of them in the near future.

I have briefly investigated Thrift and ProtoBuf but I’m still not sure that I will gain that much by switching from JSON. Also found something called BSON which is binary JSON. I think the next step will be to add some more parameters, wind direction is probably the hardest since I will have to draw the arrows myself.

Improving Google Maps polygons with b-splines

Posted on 2011-07-30 by Johan

Google Maps is great, you get an extremely nice background map for free. I know that there are alternatives (Bing, OpenLayers, etc) out there but since I’m running Google App Engine it seems easier to go Google all the way.

I’m plotting polygons and polylines (that’s what weather is about) and it works great but my input data is kind of sparse so the polygons look very rough.

To improve them I’m using b-splines. Found a very nice article here. I just changed the javascript so it works with lat/lon-arrays and the output is an array of google.maps.LatLng.

function bspline(lats, lons) {
    var i, t, ax, ay, bx, by, cx, cy, dx, dy, lat, lon, points;
    points = [];
    // For every point
    for (i = 2; i < lats.length - 2; i++) {
        for (t = 0; t < 1; t += 0.2) {
            ax = (-lats[i - 2] + 3 * lats[i - 1] - 3 * lats[i] + lats[i + 1]) / 6;
            ay = (-lons[i - 2] + 3 * lons[i - 1] - 3 * lons[i] + lons[i + 1]) / 6;
            bx = (lats[i - 2] - 2 * lats[i - 1] + lats[i]) / 2;
            by = (lons[i - 2] - 2 * lons[i - 1] + lons[i]) / 2;
            cx = (-lats[i - 2] + lats[i]) / 2;
            cy = (-lons[i - 2] + lons[i]) / 2;
            dx = (lats[i - 2] + 4 * lats[i - 1] + lats[i]) / 6;
            dy = (lons[i - 2] + 4 * lons[i - 1] + lons[i]) / 6;
            lat = ax * Math.pow(t + 0.1, 3) + bx * Math.pow(t + 0.1, 2) + cx * (t + 0.1) + dx;
            lon = ay * Math.pow(t + 0.1, 3) + by * Math.pow(t + 0.1, 2) + cy * (t + 0.1) + dy;
            points.push(new google.maps.LatLng(lat, lon));
        }
    }
    return points;
}

There are some more things that you have to do, the original arrays have to be extended by adding the first 2 elements at the back and the last 2 at the front. Or else the polygon will look chopped.

Also the first point may have to be added at the end to close a polyline, a polygon will close itself.

This is what the picture looks like without b-splines:

And with b-splines:

There are some problems with incomplete polygons (polylines that enter and exit the area) but by skipping the first and last point they look ok.

Next step is to use Thrift for communication instead of JSON. Don’t know if it will actually make any difference but I have promised myself to learn either Google Protocol Buffers or Thrift.

Google App Engine ReferenceProperty and HTML5 local storage

Posted on 2011-07-22 by Johan

The best thing with my job is that I work with the same things that I can spend hours doing in my free time. Too bad you don’t have 40 hours a week free time.

It’s been a while but I have finally made som progress.

I had some troubles with BigTable (the database that you use in Google App Engine). I put pretty large arrays with weather data in db.BlobProperty but when I read this back from the database GAE ran out of memory, even if I didn’t touch the blob. After reading up on this I found out that I had to use db.ReferenceProperty.

As always the manual is not that clear so here is some example code:

class ForecastData(db.Model):
    values = db.BlobProperty()

class Forecast(db.Model):
  firstGridPoint = db.GeoPtProperty()
  lastGridPoint = db.GeoPtProperty()
  increment = db.FloatProperty()
  parameter = db.StringProperty()
  forecast_data = db.ReferenceProperty(ForecastData)
  reference_time = db.DateTimeProperty()
  forecast_time = db.DateTimeProperty()
  insert_time = db.DateTimeProperty(auto_now_add=True)

I put my blob in a separate model and referenced it with a db.ReferenceProperty(ModelName). Below is an example for putting data in the Data Store.

    # Create the data object
    forecast_data = ForecastData()
    forecast_data.values = values

    # Put in in the database
    forecast_data = forecast_data.put()

    # Create the forecast object
    forecast = Forecast()
    # Reference the data (forecast_data is a key)
    forecast.forecast_data = forecast_data

And getting the data is done like this:

query = db.GqlQuery("SELECT * from Forecast where forecast_time=:1", forecast_time)
forecast = query.fetch(1)
if forecast:
    forecast_data = Forecast.forecast_data.get_value_for_datastore(forecast[0])
    forecast = ForecastData.get(forecast_data).values

I get the forecast object from the database with a GQL query. The referenced property can be fetched with the get_value_for_datastore method.

After this the application is much faster.

To minimize the data transfered I’m using HTML5 local storage (a very good guide to html5 can be found here).

To put something in the local storage:

window.localStorage.setItem('key', value);

and to get it back (even if the browser have been closed):

window.localStorage.getItem('key');

This is a very simple key/value store. Other useful commands are clear() which clears all saved values.

I’m hoping to launch the site for others to try out very soon but I want to get some more features in place.

Until then here is an up-to-date screenshot:

Solving the UTC problem again and again… (Setting default time zone in JVM)

Posted on 2011-07-06 by Johan

I the world of weather everything is done according to UTC and this causes problems, time zones always does.

Yesterday I was debugging an error (not related to weather) in a Spring/GWT/Hibernate app where we are scheduling things. Everything works perfect in the development and test environments but as soon as it is deployed to EC2 it fails. I had a hunch that this was time zone related (EC2 servers on Ireland). Some debugging code was added and the problem was found.

Right now the swedish time differs 2 hours because of daylight saving time so solving the problem by just subtracting hours will not work in 3 months.

The solution was easier than expected, we’re running Tomcat and all I had to do was to add a JAVA_OPT (http://muzso.hu/node/3167).

export JAVA_OPTS="-Duser.timezone=Europe/Stockholm"

Of course this will not work when the customer decides to let people not living in UTC+1 use the application but this is not a problem yet.

johan.karlsteen.com

Do or do not, there is no try