Running python tests on Travis's OSX workers

I, and my colleages are running tests mostly on linux, since our packages usually are clean python implementation. However, for mirakuru, which we use to govern processes in tests, and by extension pytest-dbfixtures, which has predefined process fixtures for pytest, we wanted to run tests also on osx. for a long time had option to configure osx workers and linux, however python builds on osx are kind of broken by default. has a long standing bug report for this issue, which will get 2 years old in this May. Apparently, just setting python as a language and osx among the system to test on won't work for apple's operating system.

Read more…

Creating a timestamp in python - difference between calendar.timegm and time.mktime

If think most of python programmers realise that there are two ways of getting timestamp with only the help of stdlib libraries. It's time.mktime and weirdly placed calendar.timegm.

There are two things people are probably not aware. First one can be attributed due to the fact that their docs are not being read properly, or more often skipped is that time.mktime function treats timetuple as if it were representing date and time in machine local time. This means even if you extract timetuple from your datetime object using utctimetuple method resulting timestamp will still be a given date in your system local time zone. Might not be a problem at all on production, but will be a much bigger issue on your machine, when you pull some data to analyse. calendar.timegm on the other hand, treats all input data as if it were passed in utc.

Speaking of which, none of those two functions accepts a parameter that would tell which timezone the timetuple is in. The only difference is that time.mktime returns a float and calendar.timegm returns an integer.

Second things - timegm function is faster than mktime! See the short benchmarks below:

Read more…

pytest-dbfixtures now works with python 3

Few days ago we've released a python3 compatible version of pytest-dbfixtures, a pytest plugin with fixtures and factories that can start fresh mysql, postgresql, elasticsearch, redis, rabbitmq and mongodb processes for Your tests - and clean their data afterwards. It also provides factories for fixtures if you'd need more processes of one kind or with different settings.

Unfortunately, if You've been using rabbitmq and/or mysql related fixtures, please check your tests carefuly, as these fixtures got most backward incompatible changes.

Read more…

Commit a day, will scratch Your itch away

At August's meeting I had a brief discussion with Tomek Wójcik about Envelopes's state, and those long opened issues at his issue tracker.

While thinking about what Tomek said, I remembered reading Jeff Knupp's post about whizbang project, a highly motivational post about willingness to do something - and it got me thinking... What makes project tick? Why does Django, Twisted, Celery have grown so big?

Read more…

My first bump with pypy

Just until two weeks ago, I was only reading about PyPy. My though about it was somewhat simple: that's the interpreter, that speeds up python code. I expected that some changes to python code, that's supposed to run in PyPy will be required, but I thought it will be easy to spot. I haven't been more wrong.

Read more…

Reminder to use pool_recycle for SQLAlchemy's connection to mysql

This post is a reminder to myself and maybe other that will stumble upon "Mysql server has gone away" error with SQLAlchemy.

Recently, we had problem again with MySQL and SQLAlchemy application, that it lost connection, resulting in this error. While a colleague of mine started panicking, I've started to remind myself, how did we solved it last time.

Well.. the answer is simple, we used pool_recycle because mysql closes itself stale connections.

I don't think this problem occurs with postgresql, at least I haven't seen it anywhere with postgresql, but for MySQL it's certainly worth trying first pool_recycle option, and tuning pooling in general, before franticly browsing internet for possible cause of this broad error.

mirakuru 0.2 released

Last Thursday, we released new minor version of mirakuru. Mirakuru is a helpful tools that lets add superpowers to Your tests, or maybe other scripts that need other processes to run.

Changes introduced to mirakuru 0.2 - are:

  • context managers (for both starting and stopping)
  • ability to detect if resources required by executors (TCP or HTTP) aren't already used.
  • and a fix, for killing executors started with their own shell.

Read more…

mirakuru, your tests little helper

Just yesterday, we released first version of mirakuru. Mirakuru is a helpful tools that lets add superpowers to Your tests, or maybe other scripts that need other processes to run.

With few lines of code, mirakuru guarantees, that when running, your script will start it's own database instance, or the webserver you've just wrote, and need to test it by sending real requests to it - automatically.

Read more…