Personal view on PyPy's future

Hello.

PyPy has seen some dramatic progress in the past year with each release
delivering roughly 30% speed improvements over the previous one. I think
with a roughly 3-month release cycle, this is a pretty good achievement and
you haven't seen all of it yet :) We have quite a few improvements in process,
with the current trunk already showing faster performance than 1.7, even
though only a month has passed.

It might be worth mentioning that PyPy has many facets and progress is done
by multiple people in various directions. Examples include:

• STM work done by Armin, read details


• NumPy speed improvements, done mostly by Alex Gaynor and myself


• NumPy completeness, done by a lot of people, including myself, Alex, Matti
  Picus and some others, if you're interested in this progressing faster, feel
  free to donate towards NumPy on PyPy


• Specialized object-instances, worked by Carl Friedrich Bolz and
  Lukas Diekmann and various other specialized types


• ARM JIT backend, work by David Schneider and Armin


• PPC JIT backend, work by Sven Hager and David Edelsohn, with help from
  David Schneider


• Speed improvements, done by everyone

Overall this is about 30 commits daily -- quite a bit of activity.

Things I can do professionally that can help the community

There is however another aspect of PyPy development, which is things that are
interesting, would work but potentially are useful only to some, but are
out of the current focus of core developers. Personally, I'm working full
time on PyPy, even though I did not receive any compensation for it for the
last half a year. I hope to make working on PyPy my full-time job from
both the donation-based income like numpy donations and pypy consulting.
I promise not to spend all the numpy-related donations on surfing addiction :-)

Examples of what I can professionally do include, but are not limited to:

• Faster json decoder. Fast json encoder was done and details can be found.


• Finshing matplotlib hacks to support full matplotlib and scipy.


• Make your favorite module X work on PyPy.


• Make your favorite module X faster on PyPy.

Additionally, I recently started offering consulting services for PyPy,
please consult my website for details.

Cheers,
Maciej Fijałkowski

Talent shortage? What talent shortage?

I've been recently reading a lot of articles like talent shortage in Austin
or even more ridiculous ideas, like an offshore office platform on the coast of
california to battle American immigration law.

This all sounds ridiculous to me, since I don't think there is any shortage of
talent and I know plenty of software developers who are more than capable and
will be willing to do work for you, provided few simple things on your side.
The crucial part is understanding that productivity differs vastly between
different people and sometimes people are productive not in a way that you
think they are. They also usually tend to have experience how to make
themselves productive.

Let me explain a bit where I come from. I'm one of the core developers of the
PyPy project and I consider myself relatively competent at least in several
fields of software engineering. My work on PyPy is not only technical -- there
is a lot of helping people, reviewing patches, communicating with
the team, whcih is entirely distributed across the world and timezones. The main
project contributors are or have been living in Germany, United States, Poland,
France, Sweden, South Africa, Italy and whatever I forgot. It does not really
matter. Personally, I claim PyPy has been a very successful project -- it works
reasonably well and it's a good way to speed up most of Python programs. Consult
the speed website for benchmark details. And on top of that there are
constantly very bright people coming and contributing in a non-trivial way,
even though PyPy has next to no money.

My experience with PyPy makes me quite competent in all-things-Python,
open source project management and understanding of nitty gritty details
of performance for the entire software stack, but also made me next to
unemployable for most companies. Let me explain few simple things you
can do to your company
that would make it a dream place to work for me and people like me. This list is
pretty personal, but I suppose the amount of flexibility is pretty universal
for a lot of smart geeks out there.

Cool technologies. PyPy is a really cool technology and I want to stay
bleeding edge with such stuff. That usually require at least two
working days a week to work on open source stuff. Your company probably
uses tons of open source, so why not contribute back? Your geeks would very
likely boost their productivity that way anyway, since most of the time
they will be working on tools they use for themselves.

Tooling. In PyPy we spend about 20-40% of our time on tooling, using
my very unscientific measurments. It's all, from buildbot maintenance,
testing tools to things like jitviewer. Overall it pays off, but it does
not show up in the near term productivity reports. Also, geeks love tools.

Telecommuting. Moving to a new place is hard for many people, especially
when you have to deal with all the immigration b*shit. Personally, I'm also
an outdoor person, so living in Cape Town is pretty ideal, I would not change
that to downtown New York for example. This is a deal breaker for many people,
but even if you don't know how to make people who are 100% telecommuting
productive, chances are they know and they already have years of experience
doing that. It also cuts a lot of costs and noone is thrilled spending
extra 2h a day driving somewhere.

The list of those three items was pretty much a dealbreaker for most of the
companies I chatted with over the past few years. There are companies who
hire remotely, but don't allow you to work on open source or the other way
around. Personally, if someone satisfies all three, I would be even willing
to have a serious paycut, just so I can live the way I like it. Living in
a cheaper place also simplifies a lot in this regard. I also don't fully
understand, why not. If today your company will start working with geeks that
way, it would be able to attract top talent, even if the end product of yours
is not very exciting. It's also a self-fulfilling prophecy -- chances are,
even if your product is not exciting, the work will be because you have
the best people on board. I would welcome some comments from
actual employers why they don't want to do all of that. If someone is
willing to do it, I'm also available for a chat, write to me at fijall at gmail
(yes, double l, gmail has 6-characters-minimum limit) or find me on twitter.

Cheers,
fijal