Colin WebbZola2024-02-28T00:00:00+00:00https://cjwebb.com/atom.xmlParallel Monte Carlo Simulation in Rust2024-02-28T00:00:00+00:002024-02-28T00:00:00+00:00
Unknown
https://cjwebb.com/parallel-monte-carlo-rust/<p>My last blog post was <a href="/simple-monte-carlo-in-rust">Simple Monte Carlo Simulations in Rust</a>.</p>
<p>It was single-threaded, and I quickly moved onto parallelizing it. Since the calculations were independent, until the end, it could be described as an 'embarassingly parallel' problem.</p>
Simple Monte Carlo Simulation in Rust2024-02-16T00:00:00+00:002024-02-16T00:00:00+00:00
Unknown
https://cjwebb.com/simple-monte-carlo-in-rust/<p>Monte Carlo simulations are often used model the probability of different outcomes. They work by generating random values, passing them into a model, and then averaging the results.</p>
<p>The simplest Monte Carlo simulation, and a great way to start, is by using the technique to estimate the value of Pi.</p>
Local LLM Development with Ollama2023-12-19T00:00:00+00:002023-12-19T00:00:00+00:00
Unknown
https://cjwebb.com/ollama-local-development/<p><img src="https://cjwebb.com/ollama-local-development/llama.jpg" alt="" /></p>
<p>Large Language Models are hot stuff at present. Using them, and integrating them into workflows, is still an open-area of investigation for many people and organisations.</p>
<p>I'm currently using VSCode, the <a href="https://continue.dev/">Continue</a> 'open-source autopilot' extension, and <a href="https://ollama.ai/">Ollama</a>.</p>
Using Github Actions for building Rust Docker Images on ARM 2023-11-15T00:00:00+00:002023-11-15T00:00:00+00:00
Unknown
https://cjwebb.com/rust-github-actions-on-arm/<p><img src="https://cjwebb.com/rust-github-actions-on-arm/crab_with_tools.jpg" alt="" /></p>
<p>Setting up CI for Rust with Github Actions is pretty easy, but there are a few gotchas. </p>
<p>This post will walk through setting up a basic CI pipeline for a Rust project, cross-compile to ARM, and build a Docker image.</p>
<p>I like to deploy hobby-projects on ARM as it is usually cheaper, and I have a Macbook so there's less cross-compilation to do.</p>
<p>We'll follow a pattern of building Rust, and then copying the binary into a Docker image. There are other patterns, such as building inside Docker using multi-stage builds, but there are unresolved issues with caching. This way is simpler, and works well.</p>
An Introduction to Demand Response in Electrical Energy Markets2023-07-31T00:00:00+00:002023-07-31T00:00:00+00:00
Unknown
https://cjwebb.com/intro-to-demand-response/<p><img src="https://cjwebb.com/intro-to-demand-response/karsten-wurth-0w-uTa0Xz7w-unsplash.jpg" alt="Photo by @karsten_wuerth on Unsplash" /></p>
<p>Demand Response is a mechanism to help balance an electricity grid. Typically, it involves incentives (economic or otherwise) for consumers to move their demand to when supply is plentiful. For example, to move electricity demand from peak times to off-peak times.</p>
Gravity Batteries At Home?2023-06-29T00:00:00+00:002023-06-29T00:00:00+00:00
Unknown
https://cjwebb.com/gravity-batteries-at-home/<p><img src="https://cjwebb.com/gravity-batteries-at-home/michael-barnes-ztuh3f37d1g-unsplash.jpg" alt="" /></p>
<p>Gravity batteries are a renewable energy storage mechanism, still being developed for the mainstream. In theory, they are very similar to <a href="https://en.wikipedia.org/wiki/Pumped-storage_hydroelectricity">pumped hydroelectric</a>. In times of low-demand and cheaper electricity, water is pumped up a hill into a reservoir. When electricity demand is high, and price is high, they let the water fall through a generator using the force of gravity.</p>
<p>Gravity batteries are another take on this principle; store energy when it is cheap and plentiful and generate electricity when it is not - through using gravity.</p>
AWS Lambda with Rust2023-05-17T00:00:00+00:002023-05-17T00:00:00+00:00
Unknown
https://cjwebb.com/aws-lambda-with-rust/<p>AWS Lambda has supported custom runtimes using Docker images for a long time now. Any language that can run in Docker, can run in AWS Lambda. Therefore, we can write AWS Lambda functions in Rust - a fun, and fast, language.</p>
<p>This quick guide gets you deploying a Rust Lambda using the AWS CLI.</p>
SCD via SQL2023-05-12T00:00:00+00:002023-05-12T00:00:00+00:00
Unknown
https://cjwebb.com/scd-via-sql/<p>Slowly Changing Dimensions (SCD) is a data engineering technique to track changes in a record over time. They're useful when changes are unpredictable, infrequent, and a historical vew of the data is needed.</p>
<p>SQL is ubiquitous in data engineering, and therefore it is the perfect tool to implement SCD with.</p>
Fake Data With Python2023-03-29T00:00:00+00:002023-03-29T00:00:00+00:00
Unknown
https://cjwebb.com/fake-data-with-python/<p>Real data is best, but sometimes it's not available - or it's not allowed.</p>
<p>Fake data is useful for testing, and for prototyping. It's also useful for generating data for demos, and for training models. Generating fake data is easy and quick.</p>
Why I'm Writing2023-03-20T00:00:00+00:002023-03-20T00:00:00+00:00
Unknown
https://cjwebb.com/why-im-writing/<p>Why am I writing?</p>
<p><strong>To practice communication.</strong></p>
<p>Written analysis, documentation, and thought are powerful ways to communicate. Writing forces me to think, and to structure my thoughts. A clear structure enables clear communication.</p>
Python Generators Cheatsheet2023-03-11T00:00:00+00:002023-03-11T00:00:00+00:00
Unknown
https://cjwebb.com/python-generators-cheatsheet/<p>A quick cheatsheet showing Python's generators, covering usage of 'yield', behaviour on raising Exceptions, and Generator Expressions.</p>
Interesting Things - October 20222022-10-05T00:00:00+00:002022-10-05T00:00:00+00:00
Unknown
https://cjwebb.com/interesting-things-october-2022/<p>In this edition of <strong>Interesting Things</strong> we're covering the technology of the Metaverse.</p>
<p>I've been reading Matthew Ball's <a href="https://www.goodreads.com/book/show/59064518-the-metaverse">The Metaverse: And How It Will Revolutionize Everything</a>.</p>
<p>Whether or not you believe the Metaverse is a thing that will exist, or should even exist, the technological challenges that need to be overcome are interesting.</p>
Git Stash: The Basics2022-05-20T00:00:00+00:002022-05-20T00:00:00+00:00
Unknown
https://cjwebb.com/simple-git-stash/<p>Git stash allows you to quickly push local changes to a stack-based storage area, thereby giving you a clean working state without creating commits or branches.</p>
<p>It is useful when manipulating previous commits, or prototyping with different changesets. Git stash only happens on your local machine and cannot be pushed to a remote server.</p>
Amazon Resource Names (ARNs)2022-01-26T00:00:00+00:002022-01-26T00:00:00+00:00
Unknown
https://cjwebb.com/amazon-resource-names/<p>An Amazon Resource Name (ARN) uniquely identifies a thing in AWS. This is a quick recap on how they are structured.</p>
Depth First Search vs Breadth First Search - A Recap2021-04-06T00:00:00+00:002021-04-06T00:00:00+00:00
Unknown
https://cjwebb.com/dfs-bfs-recap/<p>Depth First Search (DFS) and Breadth First Search (BFS) are two searching algorithms that traverse tree or graph data structures.</p>
<p>They are very similar in implementation and only differ in the choice of which node to visit next.</p>
Recent Books I Have Read2021-03-15T00:00:00+00:002021-03-15T00:00:00+00:00
Unknown
https://cjwebb.com/recent-book-reviews/<p>Quick reviews of three books I've recently read.</p>
<ol>
<li>
<p><a href="https://www.goodreads.com/book/show/53479927-the-practice">The Practice, by Seth Godin</a></p>
</li>
<li>
<p><a href="https://www.goodreads.com/book/show/18050143-zero-to-one">Zero to One, by Peter Thiel and Blake Masters</a></p>
</li>
<li>
<p><a href="https://www.goodreads.com/book/show/12605157-the-100-startup">The $100 Startup, by Chris Guillebeau</a></p>
</li>
</ol>
FFmpeg via Docker2020-11-08T00:00:00+00:002020-11-08T00:00:00+00:00
Unknown
https://cjwebb.com/ffmpeg-docker/<p>I have <a href="/posts/python-docker-ffmpeg-h264-mp4">previously written</a> about using <a href="https://www.ffmpeg.org/about.html">FFmpeg</a> to convert H264 into MP4, running in a Docker container.</p>
<p>Running desktop applications in Docker negates the need to install anything, and isolates processes more effectively than something like Brew would. I needed to use FFmpeg again this week, and so spent a little bit of time setting it up properly - using Docker. Jessie Frazelle wrote a post in 2015 named <a href="https://blog.jessfraz.com/post/docker-containers-on-the-desktop/">Containers on the Desktop</a> that was the inspiration for doing this.</p>
Solving Slow Tests2020-01-16T00:00:00+00:002020-01-16T00:00:00+00:00
Unknown
https://cjwebb.com/solving-slow-tests/<p>Everything has a cost. Some costs are obvious. Money costs, time costs, overhead costs. Most, but not all, apply to writing tests for your software.</p>
Rebooting AI - Further Reading2019-10-18T00:00:00+00:002019-10-18T00:00:00+00:00
Unknown
https://cjwebb.com/rebooting-ai-further-reading/<p>I recently read <a href="http://rebooting.ai/">Rebooting AI</a>, as book discussing how the recent focus on Deep Learning as the path towards Artificial General Intelligence is flawed. It was a good read, and made many interesting points. It left me wanting to learn more about other areas of AI, and luckily there was a 'Suggested Reading' list at the end.</p>
Thoughts on Ansible2019-09-07T00:00:00+00:002019-09-07T00:00:00+00:00
Unknown
https://cjwebb.com/thoughts-on-ansible/<p>I like Ansible, but it has flaws. I'm going to assume you know what <a href="https://en.wikipedia.org/wiki/Ansible_(software)">Ansible</a> is, and that you have used it before.</p>
<p>Basic Ansible is amazing, compared to the alternatives. When the alternative is half-baked bash scripts that rely on special environmental state, and obscure notation that is used infrequently, it is not hard to better it. I've used other automation tools too. <a href="https://en.wikipedia.org/wiki/Puppet_(company)#Puppet">Puppet</a> is the other automation tooling that I've spent considerable time with. Puppet and I did not agree with each other as it required me to learn hundreds of special syntax rules almost immediately.</p>
Hakyll Live Reload2018-05-16T00:00:00+00:002018-05-16T00:00:00+00:00
Unknown
https://cjwebb.com/hakyll-livereload/<p>I use <a href="http://nitoyon.github.io/livereloadx/">LiveReloadX</a> to automatically reload the browser as I work on my Hakyll-generated blog. LiveReloadX, and most other automatic reloading utilities work by watching the file system and notifying something in the browser to refresh the page.</p>
Python, Docker, and FFmpeg to convert H.264 to MP42018-04-09T00:00:00+00:002018-04-09T00:00:00+00:00
Unknown
https://cjwebb.com/python-docker-ffmpeg-h264-mp4/<p>H264 is a codec. MP4 is a file container, which can contain H264. If you want to display H264 video on a web browser, it needs muxing into MP4 or another file container format.</p>
Higher Kinded Play Framework2018-03-04T00:00:00+00:002018-03-04T00:00:00+00:00
Unknown
https://cjwebb.com/higher-kinded-play-framework/<p>Higher order functions are functions that accept other functions as parameters. Higher kinded types are types that can accept other types as parameters. In Scala, the syntax <code>F[_]</code> is used to express that <code>F</code> is a higher-kinded type that can accept any type.</p>
TL;DR Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning2017-12-05T00:00:00+00:002017-12-05T00:00:00+00:00
Unknown
https://cjwebb.com/tldr-neural-network-dynamics/<p>This is a <a href="https://en.wikipedia.org/wiki/Wikipedia:Too_long;_didn%27t_read">Too Long; Didn't Read</a> summary of the paper <a href="https://arxiv.org/abs/1708.02596">Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning</a>. Inspired by the <a href="http://bair.berkeley.edu/blog/2017/11/30/model-based-rl/">blog post on the same paper</a> published by the Berkeley Artifical Intelligence Research (BAIR) Lab last week.</p>
Functional Programming with the Play Framework2017-12-01T00:00:00+00:002017-12-01T00:00:00+00:00
Unknown
https://cjwebb.com/functional-play-framework/<p>Three ways to do functional programming when working with the Play Framework.</p>
Moving to Hakyll2017-07-17T00:00:00+00:002017-07-17T00:00:00+00:00
Unknown
https://cjwebb.com/moving-to-hakyll/<p>My blog previously used <a href="http://octopress.org/">Octopress</a>, and I had wanted to stop using it for a while. There is nothing wrong with Octopress; it just didn't fit well with me. It has a lot of features, so seems overcomplicated for my use-case. It's also written in Ruby.</p>
DynamoDB with Scala Macros2017-01-02T00:00:00+00:002017-01-02T00:00:00+00:00
Unknown
https://cjwebb.com/dynamodb-scala-macros/<p>I've spent the last year using <a href="https://aws.amazon.com/documentation/dynamodb/">AWS DynamoDB</a> at work. When we initially searched for a Scala client for DynamoDB, we had the following criteria:</p>
<ul>
<li>Good for Scala beginners</li>
<li>Up to date</li>
<li>Well documented</li>
</ul>
<p>Sadly, none of the Scala libraries available at that time matched all three criteria. The most up-to-date libraries were not suitable for a team starting out with Scala and DynamoDB. The most beginner-friendly libraries were out-of-date, and most only had superficial documentation.</p>
Getting Started with Haskell's Warp2016-12-16T00:00:00+00:002016-12-16T00:00:00+00:00
Unknown
https://cjwebb.com/getting-started-with-haskells-warp/<p>I recently started playing with Haskell's <a href="http://www.aosabook.org/en/posa/warp.html">Warp</a> in my effort to learn Haskell. Warp is small and fast web server, and doesn't come bundled with much. It also has no "magic" in it, which I think is a very good thing.</p>
Learning Akka Streams2016-06-28T00:00:00+00:002016-06-28T00:00:00+00:00
Unknown
https://cjwebb.com/learning-akka-streams/<p>This blog post differs from my usual ones; I'm writing it as I learn something. As such, it is more of a story that contains errors and misunderstanding than a factual blog post.</p>
The Virtues of Side Projects2016-06-15T00:00:00+00:002016-06-15T00:00:00+00:00
Unknown
https://cjwebb.com/the-virtues-of-side-projects/<p>I write software for a living, and most of my side projects are software based too. I view side projects as a tool for learning. Learning by doing.</p>
Using Elm in Octopress2016-01-01T00:00:00+00:002016-01-01T00:00:00+00:00
Unknown
https://cjwebb.com/elm-in-octopress/<p><a href="http://elm-lang.org/">Elm</a> is a functional programming language aimed at the browser. It aims to replace all HTML, CSS, and JavaScript code. It borrows a lot from Haskell, and promises that if your Elm code compiles, it will run without exceptions.</p>
Using Play-Framework's PathBindable2015-06-23T00:00:00+00:002015-06-23T00:00:00+00:00
Unknown
https://cjwebb.com/play-framework-path-binders/<p>Using custom types in <a href="https://www.playframework.com/documentation/2.4.x/ScalaRouting">Play Frameworkâs routes file</a> is a major win, and is not something obviously supported.</p>
Cassandra TTL Is Per Column2015-03-02T00:00:00+00:002015-03-02T00:00:00+00:00
Unknown
https://cjwebb.com/cassandra-ttl-is-per-column/<p><a href="http://cassandra.apache.org/">Cassandra</a> Time-To-Live (TTL) is decribed in the <a href="http://www.datastax.com/documentation/cql/3.0/cql/cql_using/use_ttl_t.html">Datastax documentation</a>. This blog post briefly explores it to demonstrate that TTL is set per column, and not per row.</p>
ScalaTest and Twitter Futures2015-02-02T00:00:00+00:002015-02-02T00:00:00+00:00
Unknown
https://cjwebb.com/scalatest-futures/<p>Scala has nice abstractions for asynchronous code. However, writing tests for that code sometimes results in an ugly, unreadable mess. Fortunately, ScalaTest has built-in support for testing Futures, in addition to utilities for other types of asynchronous testing, such as polling and test-probes.</p>
Mock Responses and Iterate2015-01-20T00:00:00+00:002015-01-20T00:00:00+00:00
Unknown
https://cjwebb.com/canned-responses/<p>'Agile' may now be an overloaded term abused by all kinds of people, but <a href="http://agilemanifesto.org/">the original manifesto</a> is actually still quite relevant. The second point is:</p>
<blockquote>
<p>Working software over comprehensive documentation</p>
</blockquote>
<p>This is useful to embrace in many circumstances, and can be expanded to:</p>
<blockquote>
<p>You should intially favour building software over documenting it. Comprehensive documentation can come later.</p>
</blockquote>
#Code20142014-12-31T00:00:00+00:002014-12-31T00:00:00+00:00
Unknown
https://cjwebb.com/code2014/<p>As the end of the year approaches, Twitter has a hash-tag named <a href="https://twitter.com/search?src=typd&q=%23code2014">#Code2014</a> that encourages people to tweet which programming languages they've been using over the last 12 months.</p>
Monoids2014-10-02T00:00:00+00:002014-10-02T00:00:00+00:00
Unknown
https://cjwebb.com/monoids/<blockquote>
<p>In abstract algebra, a branch of mathematics, a monoid is an algebraic structure with a single associative binary operation and an identity element. Monoids are studied in semigroup theory as they are semigroups with identity.</p>
</blockquote>
<p><a href="http://en.wikipedia.org/wiki/Monoid">Wikipedia has the above definition of a Monoid</a>. This blog post will desconstruct that definition to simply describe what a monoid is, and why you would want to use one.</p>
Aggregation Services in Node.js2014-07-16T00:00:00+00:002014-07-16T00:00:00+00:00
Unknown
https://cjwebb.com/aggregation-services-in-nodejs/<p><a href="http://cjwebb.github.io/blog/2014/03/26/aggregation-service-using-play-json/">My previous blog post</a> talked about building Aggregation Services using Play-JSON. In it, I mentioned that Aggregation Services using JavaScript might be quite nice. As JSON is native to JavaScript, you might expect manipulating JSON in JavaScript to be incredibly simple. And you would be correct!</p>
Aggregation Services using Play JSON2014-03-26T00:00:00+00:002014-03-26T00:00:00+00:00
Unknown
https://cjwebb.com/aggregation-service-using-play-json/<p>Aggregation services (sometimes known as Composite or Hydration services) are useful when working in SOA. In SOA, services are responsible for discrete objects and collections, yet still often need to reference other object or collections controlled by another service. This is done via referencing Ids. In order to display something useful to the user it is necessary to lookup data from multiple sources and aggregate them together.</p>
Fibonacci Numbers in Scala2013-10-30T00:00:00+00:002013-10-30T00:00:00+00:00
Unknown
https://cjwebb.com/fibonacci-numbers-in-scala/<p>Working out the <a href="http://en.wikipedia.org/wiki/Fibonacci_number">Fibonacci numbers</a> are a standard exercise when learning a programming language, or just refreshing your knowledge. This is a recap of various ways to calculate them in Scala. <a href="https://gist.github.com/cjwebb/7239843">All the code is available in a handy gist</a>.</p>
Akka Camel and ActiveMQ2013-09-01T00:00:00+00:002013-09-01T00:00:00+00:00
Unknown
https://cjwebb.com/akka-camel-and-activemq/<p>I've been using <a href="http://doc.akka.io/docs/akka/snapshot/scala/camel.html" title="Akka Camel">Akka Camel</a> and <a href="http://activemq.apache.org/" title="Active MQ">ActiveMQ</a> recently, as part of a delayed worker-queue system. Given the lack of good googleable information about combining the two, I thought it would be useful if I explained briefly how to get Akka Camel and ActiveMQ to work together in the form of a quick example.</p>
Testing in Scala with DSLs2013-07-20T00:00:00+00:002013-07-20T00:00:00+00:00
Unknown
https://cjwebb.com/testing-in-scala-with-dsls/<p>Using a DSL to write a test can prove to be useful, especially when there are lots of prerequisites, or the problem is complex. Having a test that is incredibly readable reduces complexity overhead and aids reader comprehension. Any tests that require the reader to retain a mental map could benefit from a DSL.</p>
Better Assertions with ScalaTest2013-05-27T00:00:00+00:002013-05-27T00:00:00+00:00
Unknown
https://cjwebb.com/better-assertions-with-scalatest/<p>Scala has three different frameworks for writing unit tests, JUnit, <a href="http://etorreborre.github.io/specs2/guide/org.specs2.guide.QuickStart.html" title="Specs2">Specs2</a> and <a href="http://www.scalatest.org/" title="ScalaTest">ScalaTest</a>. I have mainly been using ScalaTest since I started to learn Scala about four months ago. One of the areas I have been concentrating on is how to write good assertions. Specifically, ones that yield easily diagnosable error messages when they fail.</p>
Using MongoDB TTL Collections to Manage User Registrations2012-12-17T00:00:00+00:002012-12-17T00:00:00+00:00
Unknown
https://cjwebb.com/mongodb-ttl-collections/<p><a href="https://docs.mongodb.com/manual/tutorial/expire-data/">TTL Collections</a> were introduced in MongoDB 2.2. You are now able to create an index which triggers the deletion of a document after the Time-To-Live (TTL) time passes.</p>
How to JUnit Test a Private Method2010-10-31T00:00:00+00:002010-10-31T00:00:00+00:00
Unknown
https://cjwebb.com/junit-test-private-method/<p>I was recently writing a cache for MP3s at work, and I needed to test some private methods. Testing a private method in Java is doable using reflection.</p>