Notes on the Julia programming language
I'm using the Julia programming language for my PhD work and this page records various notes as an aide-mémoire. I acknowledge that some of the material is opinionated, but I'm happy to accept corrections or contributions.
Download and install the Julia current release. Any tutorials by David Sanders are good, e.g. An Invitation to Julia. There are some books (see Learning Julia), but the Julia documentation arguably serves as a better resource.
There are lots of options including Juno and Visual Studio Code, but I'm just using Emacs and julia-mode.
I really like IJulia (Jupyter notebooks) and tend to have one per experiment / investigation, printing them out and discussing them at supervisory meetings.
It's easy to call Python libraries from Julia, and this is a useful trap door if you need a specialist library. See PyCall.jl and Conda.jl.
I like to set
export CONDA_JL_VERSION="3" in my shell, prior to installing the Conda.jl package
to pick up Python 3 instead of Python 2.
Conda.ROOTENV within Julia gives the directory of
the underlying Python environment. This allows you to hop out
to a shell,
activate the environment and
odd packages like
using PyCall @pyimport pynmea2 pynmea2.parse(mystring)
I often start protototyping code in an IJulia notebook. When an idea
starts to come together I move it into a separate text file in
a directory I call
includes. I can then reuse it across
notebooks and scripts with:
Once I have a quorum of code on a particular topic and it starts to
feel like a reusable library, I create a package using PkgDev.
I usually move that package directory from
~/.julia/vX.X to my regular working
area and then link back to it using
ln -s ~/Dropbox/phd/julia/mypackage.jl mypackage.
It's a good idea to make the real directory name
something.jl and link just
Pkg.test facility doesn't seem to work.
There are lots of plotting libraries in Julia. My strategy is to use Plots.jl whenever I can and fall back to PyPlot for certain edge cases.
plotlyjs() back end is particulalrly good for
interactive, exploratory programming. You might find that it
blows-up for large data sets, in which case try changing the backend
interp2 is a common thing in MATLAB
is the Julia equivalent. This code is similar:
using Interpolations function interp1(X, V, Xq) knots = (X,) itp = interpolate(knots, V, Gridded(Linear())) itp[Xq] end function interp2(X, Y, V, Xq, Yq) knots = (X,Y) itp = interpolate(knots, V, Gridded(Linear())) itp[Xq, Yq] end
Julia can be used to make shell scripts, using the shebang line below:
This can be useful in large projects where it allows for Make to be used to build products incrementally. Beware, the Julia start up time can be large in comparison to the run time of your script so it can be very inefficient to work with small files and tasks.
The output of info(), warn() and error() is sent to STDERR instead of STDOUT making them ideal mechanisms for instrumenting and logging scripts, without messing up output.
$ ./script.jl > output.txt 2> log.txt
The above results in two files, output.txt with everything your printed and log.txt with everything written by info, warn and error.
On Ubuntu 18.04 LTS
If you use
ImageMagick.jl directly or indirectly,
please note that the Ubuntu supplied
is old and caused me lots of problems with cache
You can update by grabbing ImageMagick-6.9.9-51.tar.xz
and unpacking it under
/usr/local/src then doing
sudo checkinstall dance. After that you'll need to
Pkg.build("ImageMagick) and check the version is 6.9.9 with
Julia works well on FreeBSD, but installation and setup sometimes need a little more thought. There is no Conda or other installer, so you must install platform libraries yourself. Packages are usually tested on Windows, Mac and Linux rather than FreeBSD, so be prepared for a bit of fixing up now and again.
Assuming FreeBSD 11.1 STABLE, julia 6.2 can be installed
pkg install julia. It's hard to
go far in Julia without stumbling on some Python. The trick
is to install Python and PyCall first:
pkg install python3
Then, in Julia:
In general it's best to use the Pkg system and avoid
Pip for installing Python libraries on FreeBSD, but
py36-matpotlib is quite old and
requires a newer version, so:
pkg install py36-matplotlib
pkg install py36-pip pip-3.6 install --user matplotlib
For MbedTLS.jl you'll need to:
pkg install mbedtls
IJulia needs some coaxing, see Fix to allow installation on FreeBSD 11.1 STABLE.
Compared to Python
Whilst Python code can be optimised, I find most Julia code to be more performant with no effort (although I acknowledge that this isn't always the case). For me, the switch to Julia meant that I didn't have to move to a high performance compute cluster from the convenience of a couple of PCs in the corner of my study.
Most of the time, Julia just feels nicer to use, and that's important when you spend a large proportion of your life staring at a REPL.
The whole Python 2 or Python 3 thing is just irritating.
That said, there is still a lot of Python code under the Julia hood (e.g. IJulia) so don't knock it!
Compared to MATLAB
MATLAB usage is widespread amongst scientists for good reason; it's established and has very high quality, proven libraries. In many ways it is the gold standard for scientific, exploratory programming.
Unfortunately, a lot of graduate student science is
just grunt work. Reading legacy file formats and
big data is a big part of
that. MATLAB can do it, but it isn't very elegant. I
also found problems with compatability between
versions. No two scientists seem to use the same
it worked on my machine is a
common cry. Upgrading a machine is often considered
too risky or too expensive.
I found that MATLAB programmers can readily understand Julia code. Googling for a MATLAB solution to a problem often turns up a function name, and that can be the key to finding an equivalent Julia solution.
Julia is free and open source. The barrier to entry is much lower and that's got to be a good thing for reprodicible science.
Niggles and problems
Because the package system uses git, the local library directory
~/.julia ends up being
large, mine is currently about 3.7G, and time consuming to set up on a new machine. Fortunately,
there is a new package manager. Pkg3 in the works.
Julia is a Lisp-1 so naming variables so that they don't clash with functions can sometimes be annoying, e.g
julia> wheels = wheels(mycar) ERROR: invalid redefinition of constant wheels
Some people worry that
you're using a language which hasn't even reached 1.0. In practice
this has never really caused me a problem, the language developers are dilligent about
deprecating old syntax.
Compiler lag: You
Pkg.update(), then the next time you come to use an
updated package you have to sit and wait for it to compile. This gives the impression
tha Julia is slow, particularly when working interactively. Once compiled, it's typically
Updating packages can cause alarming WARNING and ERROR messages. Don't
worry about the warnings, if you get errors then
is almost always a cure.
Is there a way to ask
whocalls(f) where f is some function?
I'd like some refactoring tools.
SLIME support in Emacs would be great. I have a proof of concept SWANK server.
Last modified: Thu Jun 20:50:17 UTC 2018 by reb