Notes on the Julia programming language

Rob Blackwell

Introduction

I'm using the Julia programming language for my PhD work and this page records various notes as an aide-mémoire. I acknowledge that some of the material is opinionated, but I'm happy to accept corrections or contributions.

Getting Started

Download and install the Julia current release. Any tutorials by David Sanders are good, e.g. An Invitation to Julia. There are some books (see Learning Julia), but the Julia documentation arguably serves as a better resource.

Tools

There are lots of options including Juno and Visual Studio Code, but I'm just using Emacs and julia-mode.

I really like IJulia (Jupyter notebooks) and tend to have one per experiment / investigation, printing them out and discussing them at supervisory meetings.

Python support

It's easy to call Python libraries from Julia, and this is a useful trap door if you need a specialist library. See PyCall.jl and Conda.jl.

I like to set export CONDA_JL_VERSION="3" in my shell, prior to installing the Conda.jl package to pick up Python 3 instead of Python 2.

Examining Conda.ROOTENV within Julia gives the directory of the underlying Python environment. This allows you to hop out to a shell, activate the environment and pip install odd packages like pynmea2.

using PyCall
@pyimport pynmea2
pynmea2.parse(mystring)
		

Code reuse

I often start protototyping code in an IJulia notebook. When an idea starts to come together I move it into a separate text file in a directory I call includes. I can then reuse it across notebooks and scripts with: include(expanduser("~/Dropbox/phd/julia/include/plotsutils.jl"));

Once I have a quorum of code on a particular topic and it starts to feel like a reusable library, I create a package using PkgDev. I usually move that package directory from ~/.julia/vX.X to my regular working area and then link back to it using ln -s ~/Dropbox/phd/julia/mypackage.jl mypackage. It's a good idea to make the real directory name something.jl and link just something otherwise the Pkg.test facility doesn't seem to work.

Plotting

There are lots of plotting libraries in Julia. My strategy is to use Plots.jl whenever I can and fall back to PyPlot for certain edge cases.

IJulia with Plots.jl and the plotlyjs() back end is particulalrly good for interactive, exploratory programming. You might find that it blows-up for large data sets, in which case try changing the backend to pyplot().

Interpolation

Interpolation via interp1 and interp2 is a common thing in MATLAB and Interpolations.jl is the Julia equivalent. This code is similar:

using Interpolations

function interp1(X, V, Xq)
    knots = (X,)
    itp = interpolate(knots, V, Gridded(Linear()))
    itp[Xq]
end

function interp2(X, Y, V, Xq, Yq)
    knots = (X,Y)
    itp = interpolate(knots, V, Gridded(Linear()))
    itp[Xq, Yq]
end
	    

Scripting

Julia can be used to make shell scripts, using the shebang line below:

#!/usr/bin/env julia
	    

This can be useful in large projects where it allows for Make to be used to build products incrementally. Beware, the Julia start up time can be large in comparison to the run time of your script so it can be very inefficient to work with small files and tasks.

Logging

The output of info(), warn() and error() is sent to STDERR instead of STDOUT making them ideal mechanisms for instrumenting and logging scripts, without messing up output.

$ ./script.jl > output.txt 2> log.txt
	    

The above results in two files, output.txt with everything your printed and log.txt with everything written by info, warn and error.

On Ubuntu 18.04 LTS

If you use ImageMagick.jl directly or indirectly, please note that the Ubuntu supplied libMagickWand is old and caused me lots of problems with cache exhaustion.

You can update by grabbing ImageMagick-6.9.9-51.tar.xz and unpacking it under /usr/local/src then doing the usual ./configure, make, sudo checkinstall dance. After that you'll need to Pkg.build("ImageMagick) and check the version is 6.9.9 with ImageMagick.libversion().

On FreeBSD

Julia works well on FreeBSD, but installation and setup sometimes need a little more thought. There is no Conda or other installer, so you must install platform libraries yourself. Packages are usually tested on Windows, Mac and Linux rather than FreeBSD, so be prepared for a bit of fixing up now and again.

Assuming FreeBSD 11.1 STABLE, julia 6.2 can be installed using pkg install julia. It's hard to go far in Julia without stumbling on some Python. The trick is to install Python and PyCall first:

pkg install python3
	    

Then, in Julia:

ENV["PYTHON"]="python"
Pkg.add("PyCall")
	    

In general it's best to use the Pkg system and avoid Pip for installing Python libraries on FreeBSD, but unfortunately py36-matpotlib is quite old and PyPlots requires a newer version, so:

pkg install py36-matplotlib
	    

Then:

pkg install py36-pip
pip-3.6 install --user matplotlib
	    

For MbedTLS.jl you'll need to:

pkg install mbedtls
	    

IJulia needs some coaxing, see Fix to allow installation on FreeBSD 11.1 STABLE.

Compared to Python

Whilst Python code can be optimised, I find most Julia code to be more performant with no effort (although I acknowledge that this isn't always the case). For me, the switch to Julia meant that I didn't have to move to a high performance compute cluster from the convenience of a couple of PCs in the corner of my study.

Most of the time, Julia just feels nicer to use, and that's important when you spend a large proportion of your life staring at a REPL.

The whole Python 2 or Python 3 thing is just irritating.

That said, there is still a lot of Python code under the Julia hood (e.g. IJulia) so don't knock it!

Compared to MATLAB

MATLAB usage is widespread amongst scientists for good reason; it's established and has very high quality, proven libraries. In many ways it is the gold standard for scientific, exploratory programming.

Unfortunately, a lot of graduate student science is just grunt work. Reading legacy file formats and dealing with big data is a big part of that. MATLAB can do it, but it isn't very elegant. I also found problems with compatability between versions. No two scientists seem to use the same version and it worked on my machine is a common cry. Upgrading a machine is often considered too risky or too expensive.

I found that MATLAB programmers can readily understand Julia code. Googling for a MATLAB solution to a problem often turns up a function name, and that can be the key to finding an equivalent Julia solution.

Julia is free and open source. The barrier to entry is much lower and that's got to be a good thing for reprodicible science.

Niggles and problems

Because the package system uses git, the local library directory ~/.julia ends up being large, mine is currently about 3.7G, and time consuming to set up on a new machine. Fortunately, there is a new package manager. Pkg3 in the works.

Julia is a Lisp-1 so naming variables so that they don't clash with functions can sometimes be annoying, e.g

julia> wheels = wheels(mycar)
ERROR: invalid redefinition of constant wheels

Some people worry that you're using a language which hasn't even reached 1.0. In practice this has never really caused me a problem, the language developers are dilligent about deprecating old syntax.

Compiler lag: You Pkg.update(), then the next time you come to use an updated package you have to sit and wait for it to compile. This gives the impression tha Julia is slow, particularly when working interactively. Once compiled, it's typically very fast.

Updating packages can cause alarming WARNING and ERROR messages. Don't worry about the warnings, if you get errors then Pkg.build("somepackage") is almost always a cure.

Wishlist

Is there a way to ask whocalls(f) where f is some function?

I'd like some refactoring tools.

SLIME support in Emacs would be great. I have a proof of concept SWANK server.

Last modified: Thu Jun 20:50:17 UTC 2018 by reb