Colin Webb

AWS Bedrock with Rust

2024-04-14T00:00:00+00:00

AWS Bedrock is a fully-managed service that provides LLM interactions and has models from lots of leading organisations</a>. In my opinion, it's a great way to get started with LLMs if you're using AWS since there's no worry about infrastructure scaling.

This post is a short description of how to get AWS Bedrock responding to your Rust code. It's a simple example, but should point the way forward for more complex interactions.

Parallel Monte Carlo Simulation in Rust

2024-02-28T00:00:00+00:00

My last blog post was Simple Monte Carlo Simulations in Rust</a>.

It was single-threaded, and I quickly moved onto parallelizing it. Since the calculations were independent, until the end, it could be described as an 'embarassingly parallel' problem.

Simple Monte Carlo Simulation in Rust

2024-02-16T00:00:00+00:00

Monte Carlo simulations are often used model the probability of different outcomes. They work by generating random values, passing them into a model, and then averaging the results.

The simplest Monte Carlo simulation, and a great way to start, is by using the technique to estimate the value of Pi.

Local LLM Development with Ollama

2023-12-19T00:00:00+00:00

Large Language Models are hot stuff at present. Using them, and integrating them into workflows, is still an open-area of investigation for many people and organisations.

I'm currently using VSCode, the Continue</a> 'open-source autopilot' extension, and Ollama</a>.

Using Github Actions for building Rust Docker Images on ARM

2023-11-15T00:00:00+00:00

Setting up CI for Rust with Github Actions is pretty easy, but there are a few gotchas.

This post will walk through setting up a basic CI pipeline for a Rust project, cross-compile to ARM, and build a Docker image.

I like to deploy hobby-projects on ARM as it is usually cheaper, and I have a Macbook so there's less cross-compilation to do.

We'll follow a pattern of building Rust, and then copying the binary into a Docker image. There are other patterns, such as building inside Docker using multi-stage builds, but there are unresolved issues with caching. This way is simpler, and works well.

An Introduction to Demand Response in Electrical Energy Markets

2023-07-31T00:00:00+00:00

Demand Response is a mechanism to help balance an electricity grid. Typically, it involves incentives (economic or otherwise) for consumers to move their demand to when supply is plentiful. For example, to move electricity demand from peak times to off-peak times.

Gravity Batteries At Home?

2023-06-29T00:00:00+00:00

Gravity batteries are a renewable energy storage mechanism, still being developed for the mainstream. In theory, they are very similar to pumped hydroelectric</a>. In times of low-demand and cheaper electricity, water is pumped up a hill into a reservoir. When electricity demand is high, and price is high, they let the water fall through a generator using the force of gravity.

Gravity batteries are another take on this principle; store energy when it is cheap and plentiful and generate electricity when it is not - through using gravity.

AWS Lambda with Rust

2023-05-17T00:00:00+00:00

AWS Lambda has supported custom runtimes using Docker images for a long time now. Any language that can run in Docker, can run in AWS Lambda. Therefore, we can write AWS Lambda functions in Rust - a fun, and fast, language.

This quick guide gets you deploying a Rust Lambda using the AWS CLI.

SCD via SQL

2023-05-12T00:00:00+00:00

Slowly Changing Dimensions (SCD) is a data engineering technique to track changes in a record over time. They're useful when changes are unpredictable, infrequent, and a historical vew of the data is needed.

SQL is ubiquitous in data engineering, and therefore it is the perfect tool to implement SCD with.

Fake Data With Python

2023-03-29T00:00:00+00:00

Real data is best, but sometimes it's not available - or it's not allowed.

Fake data is useful for testing, and for prototyping. It's also useful for generating data for demos, and for training models. Generating fake data is easy and quick.

Why I'm Writing

2023-03-20T00:00:00+00:00

Why am I writing?

To practice communication.

Written analysis, documentation, and thought are powerful ways to communicate. Writing forces me to think, and to structure my thoughts. A clear structure enables clear communication.

Python Generators Cheatsheet

2023-03-11T00:00:00+00:00

A quick cheatsheet showing Python's generators, covering usage of 'yield', behaviour on raising Exceptions, and Generator Expressions.

Interesting Things - October 2022

2022-10-05T00:00:00+00:00

In this edition of Interesting Things we're covering the technology of the Metaverse.
I've been reading Matthew Ball's The Metaverse: And How It Will Revolutionize Everything</a>.
Whether or not you believe the Metaverse is a thing that will exist, or should even exist, the technological challenges that need to be overcome are interesting.

Git Stash: The Basics

2022-05-20T00:00:00+00:00

Git stash allows you to quickly push local changes to a stack-based storage area, thereby giving you a clean working state without creating commits or branches.
It is useful when manipulating previous commits, or prototyping with different changesets. Git stash only happens on your local machine and cannot be pushed to a remote server.

Amazon Resource Names (ARNs)

2022-01-26T00:00:00+00:00

An Amazon Resource Name (ARN) uniquely identifies a thing in AWS. This is a quick recap on how they are structured.

Depth First Search vs Breadth First Search - A Recap

2021-04-06T00:00:00+00:00

Depth First Search (DFS) and Breadth First Search (BFS) are two searching algorithms that traverse tree or graph data structures.
They are very similar in implementation and only differ in the choice of which node to visit next.

Recent Books I Have Read

2021-03-15T00:00:00+00:00

Quick reviews of three books I've recently read.

The Practice, by Seth Godin</a> </li>

Zero to One, by Peter Thiel and Blake Masters</a> </li>

The $100 Startup, by Chris Guillebeau</a> </li> </ol>

FFmpeg via Docker

2020-11-08T00:00:00+00:00

I have previously written</a> about using FFmpeg</a> to convert H264 into MP4, running in a Docker container.
Running desktop applications in Docker negates the need to install anything, and isolates processes more effectively than something like Brew would. I needed to use FFmpeg again this week, and so spent a little bit of time setting it up properly - using Docker. Jessie Frazelle wrote a post in 2015 named Containers on the Desktop</a> that was the inspiration for doing this.

Solving Slow Tests

2020-01-16T00:00:00+00:00

Everything has a cost. Some costs are obvious. Money costs, time costs, overhead costs. Most, but not all, apply to writing tests for your software.

Rebooting AI - Further Reading

2019-10-18T00:00:00+00:00

I recently read Rebooting AI</a>, as book discussing how the recent focus on Deep Learning as the path towards Artificial General Intelligence is flawed. It was a good read, and made many interesting points. It left me wanting to learn more about other areas of AI, and luckily there was a 'Suggested Reading' list at the end.

Thoughts on Ansible

2019-09-07T00:00:00+00:00

I like Ansible, but it has flaws. I'm going to assume you know what Ansible</a> is, and that you have used it before.
Basic Ansible is amazing, compared to the alternatives. When the alternative is half-baked bash scripts that rely on special environmental state, and obscure notation that is used infrequently, it is not hard to better it. I've used other automation tools too. Puppet</a> is the other automation tooling that I've spent considerable time with. Puppet and I did not agree with each other as it required me to learn hundreds of special syntax rules almost immediately.

Hakyll Live Reload

2018-05-16T00:00:00+00:00

I use LiveReloadX</a> to automatically reload the browser as I work on my Hakyll-generated blog. LiveReloadX, and most other automatic reloading utilities work by watching the file system and notifying something in the browser to refresh the page.

Python, Docker, and FFmpeg to convert H.264 to MP4

2018-04-09T00:00:00+00:00

H264 is a codec. MP4 is a file container, which can contain H264. If you want to display H264 video on a web browser, it needs muxing into MP4 or another file container format.

Higher Kinded Play Framework

2018-03-04T00:00:00+00:00

Higher order functions are functions that accept other functions as parameters. Higher kinded types are types that can accept other types as parameters. In Scala, the syntax F[_]</code> is used to express that F</code> is a higher-kinded type that can accept any type.
TL;DR Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning 2017-12-05T00:00:00+00:00 This is a Too Long; Didn't Read</a> summary of the paper Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning</a>. Inspired by the blog post on the same paper</a> published by the Berkeley Artifical Intelligence Research (BAIR) Lab last week. Functional Programming with the Play Framework 2017-12-01T00:00:00+00:00 Three ways to do functional programming when working with the Play Framework. Moving to Hakyll 2017-07-17T00:00:00+00:00 My blog previously used Octopress</a>, and I had wanted to stop using it for a while. There is nothing wrong with Octopress; it just didn't fit well with me. It has a lot of features, so seems overcomplicated for my use-case. It's also written in Ruby. DynamoDB with Scala Macros 2017-01-02T00:00:00+00:00 I've spent the last year using AWS DynamoDB</a> at work. When we initially searched for a Scala client for DynamoDB, we had the following criteria: Good for Scala beginners</li> Up to date</li> Well documented</li> </ul> Sadly, none of the Scala libraries available at that time matched all three criteria. The most up-to-date libraries were not suitable for a team starting out with Scala and DynamoDB. The most beginner-friendly libraries were out-of-date, and most only had superficial documentation. Getting Started with Haskell's Warp 2016-12-16T00:00:00+00:00 I recently started playing with Haskell's Warp</a> in my effort to learn Haskell. Warp is small and fast web server, and doesn't come bundled with much. It also has no "magic" in it, which I think is a very good thing. Learning Akka Streams 2016-06-28T00:00:00+00:00 This blog post differs from my usual ones; I'm writing it as I learn something. As such, it is more of a story that contains errors and misunderstanding than a factual blog post. The Virtues of Side Projects 2016-06-15T00:00:00+00:00 I write software for a living, and most of my side projects are software based too. I view side projects as a tool for learning. Learning by doing. Using Elm in Octopress 2016-01-01T00:00:00+00:00 Elm</a> is a functional programming language aimed at the browser. It aims to replace all HTML, CSS, and JavaScript code. It borrows a lot from Haskell, and promises that if your Elm code compiles, it will run without exceptions. Using Play-Framework's PathBindable 2015-06-23T00:00:00+00:00 Using custom types in Play Framework’s routes file</a> is a major win, and is not something obviously supported. Cassandra TTL Is Per Column 2015-03-02T00:00:00+00:00 Cassandra</a> Time-To-Live (TTL) is decribed in the Datastax documentation</a>. This blog post briefly explores it to demonstrate that TTL is set per column, and not per row. ScalaTest and Twitter Futures 2015-02-02T00:00:00+00:00 Scala has nice abstractions for asynchronous code. However, writing tests for that code sometimes results in an ugly, unreadable mess. Fortunately, ScalaTest has built-in support for testing Futures, in addition to utilities for other types of asynchronous testing, such as polling and test-probes. Mock Responses and Iterate 2015-01-20T00:00:00+00:00 'Agile' may now be an overloaded term abused by all kinds of people, but the original manifesto</a> is actually still quite relevant. The second point is: Working software over comprehensive documentation </blockquote> This is useful to embrace in many circumstances, and can be expanded to: You should intially favour building software over documenting it. Comprehensive documentation can come later. </blockquote> #Code2014 2014-12-31T00:00:00+00:00 As the end of the year approaches, Twitter has a hash-tag named #Code2014</a> that encourages people to tweet which programming languages they've been using over the last 12 months. Monoids 2014-10-02T00:00:00+00:00 In abstract algebra, a branch of mathematics, a monoid is an algebraic structure with a single associative binary operation and an identity element. Monoids are studied in semigroup theory as they are semigroups with identity. </blockquote> Wikipedia has the above definition of a Monoid</a>. This blog post will desconstruct that definition to simply describe what a monoid is, and why you would want to use one. Aggregation Services in Node.js 2014-07-16T00:00:00+00:00 My previous blog post</a> talked about building Aggregation Services using Play-JSON. In it, I mentioned that Aggregation Services using JavaScript might be quite nice. As JSON is native to JavaScript, you might expect manipulating JSON in JavaScript to be incredibly simple. And you would be correct! Aggregation Services using Play JSON 2014-03-26T00:00:00+00:00 Aggregation services (sometimes known as Composite or Hydration services) are useful when working in SOA. In SOA, services are responsible for discrete objects and collections, yet still often need to reference other object or collections controlled by another service. This is done via referencing Ids. In order to display something useful to the user it is necessary to lookup data from multiple sources and aggregate them together. Fibonacci Numbers in Scala 2013-10-30T00:00:00+00:00 Working out the Fibonacci numbers</a> are a standard exercise when learning a programming language, or just refreshing your knowledge. This is a recap of various ways to calculate them in Scala. All the code is available in a handy gist</a>. Akka Camel and ActiveMQ 2013-09-01T00:00:00+00:00 I've been using Akka Camel</a> and ActiveMQ</a> recently, as part of a delayed worker-queue system. Given the lack of good googleable information about combining the two, I thought it would be useful if I explained briefly how to get Akka Camel and ActiveMQ to work together in the form of a quick example. Testing in Scala with DSLs 2013-07-20T00:00:00+00:00 Using a DSL to write a test can prove to be useful, especially when there are lots of prerequisites, or the problem is complex. Having a test that is incredibly readable reduces complexity overhead and aids reader comprehension. Any tests that require the reader to retain a mental map could benefit from a DSL. Better Assertions with ScalaTest 2013-05-27T00:00:00+00:00 Scala has three different frameworks for writing unit tests, JUnit, Specs2</a> and ScalaTest</a>. I have mainly been using ScalaTest since I started to learn Scala about four months ago. One of the areas I have been concentrating on is how to write good assertions. Specifically, ones that yield easily diagnosable error messages when they fail. Using MongoDB TTL Collections to Manage User Registrations 2012-12-17T00:00:00+00:00 TTL Collections</a> were introduced in MongoDB 2.2. You are now able to create an index which triggers the deletion of a document after the Time-To-Live (TTL) time passes. How to JUnit Test a Private Method 2010-10-31T00:00:00+00:00 I was recently writing a cache for MP3s at work, and I needed to test some private methods. Testing a private method in Java is doable using reflection.