Building autoscaling CI infrastructure with Azure Kubernetes

2019-10-17 9 min read anoff

Ever wanted to create a build agent factory where you do not have to care about how many build agents you need at a given point? With this post I want to share my experience setting up a dedicated CI runner infrastructure with the Azure + Pipelines ecosystem. The main features of the solution are automated scaling, ephemeral build agents, docker based environments, minimal operation responsible and strong pay-per-use billing concepts. Continue reading

Hosting Gitea and Drone with Docker

2019-03-24 8 min read anoff

This post will walk you through setting up a self hosted git based continuous integration environment on a two machine setup - assuming you already have two virtual machines at your disposal. Using Gitea for git hosting and contribution management and Drone for docker-based build jobs, this will guide you through creating docker-compose files as well as configuring the individual services and getting SSL certificates via traefik. Docker and docker-compose knowledge is required for this tutorial. It mostly focuses on the correct configuration of all the services at play here and not explaining their basic functionality.

This tutorial uses Azure resources so some of the aspects might not be 100% applicable if you chose another infrastructure provider.

Configure Azure Active Directory as Authentication Provider for Gitea

2019-03-23 3 min read anoff

Gitea is an open source git hosting platform, written in Go forked from the Gogs project and seems to be highly inspired by GitHub. If you want to host your own git platform and want to use Azure Active Directory as an authentication provider - this post might help you.

Automated dev workflow for using Data Science VM on Azure

2018-03-22 11 min read anoff

tl;dr; I put together a bunch of scripts on Github that let you deploy a VM from your command line as well as sync code from your local directory to the VM easily to be able to use local IDE and git but execute on the powerful remote machine. Perfect for Data Science applications based around jupyter notebook. In my previous blog post I explained how to do Terraform deployment of an Azure Data Science Virtual Machine. Continue reading

Deploy Datascience infrastructure on Azure using Terraform

2018-01-23 6 min read anoff

In this article I will talk about my experience building my first infrastructure deployment using Terraform that does (a little) more than combining off-the-shelf resources. The stack we will deploy 📦 Lately I’ve been looking at a lot of Microsoft Azure services in the big data area. I am looking for something to replace a Hadoop based 🐘 data analytics environment consisting mainly of HDFS, Spark & Jupyter. The most obvious solution is to use a HDInsight cluster which is basically a managed Hadoop that you can pick in different flavours. Continue reading