Ruan Bekker
@ruanbekker
Joined February 15th, 2021
I love cloudy stuff, containers, automation, linux and sharing my knowledge with the world.
Performance Monitoring with Elastic APM
In this post we will setup a Python Flask Application which includes the APM Agent which will collect metrics, that gets pushed to the APM Server. If you have not setup the Elastic Stack with / or APM Server, you can follow this post to setup the needed.
Then we will make a bunch of HTTP Requests to our Application and will go...
2
Select One:
Weather Monitoring with Grafana and the DHT22 Sensor
In this tutorial, we will connect the DHT22 sensor to the Raspberry Pi Zero via the GPIO pins to measure temperature and humidity and visualize it with Grafana.
Then we will write a Python exporter for prometheus to expose our metrics so that we can visualize it in Grafana.
The Endgoal
The Hardware
This is how the sensor looks...
4
Select One:
Getting Started with Prometheus and Node Exporter
Prometheus is one of those awesome open source monitoring services that I simply cannot live without. Prometheus is a Time Series Database that collects metrics from services using it's exporters functionality. Prometheus has its own query language called PromQL and makes graphing epic visualiztions with services such as Grafana...
7
Select One:
AWS S3 KMS and Python for Secrets Management
So your application need to store secrets and you are looking for a home for them. In this tutorial we will see how we can use Python, S3 and KMS to build our own solution for managing secrets.
There is SSM and Secrets Manager that probably does a better job, but my mind got curious :D
High Level Goal
From a High-Level we want t...
3
Select One:
Setup a Serverless URL Shortener on AWS
Today we will setup a Serverless URL Shortener using API Gateway, Lambda with Python and DynamoDB.
Overview
The service that we will be creating, will shorten URLs via our API which will create an entry on DynamoDB. When a GET method is performed on the shortened URL, a GetItem is executed on DynamoDB to get the Long URL and a 3...
3
Select One:
Using Python RQ for Task Queues in Python
This is a getting started on python-rq tutorial and I will demonstrate how to work with asynchronous tasks using python redis queue (python-rq).
What will we be doing
We want a client to submit 1000's of jobs in a non-blocking asynchronous fashion, and then we will have workers which will consume these jobs from our redis queue,...
5
Select One:
Gitlab CICD Pipeline to Deploy your Python Flask API with Postgres on Heroku
Today we will build a Restful API using Python Flask, SQLAlchemy using Postgres as our Database, testing using Python Unittest, a CI/CD Pipeline on Gitlab, and Deployment to Heroku.
Should you be interested in setting up a Custom Gitlab Runner on Your Own Server for Gitlab CI have a look at the embedded post.
Heroku
If you don'...
4
Select One:
Capturing 54 Million Passwords with a Docker SSH Honeypot
The last couple of days I observed thousands of SSH Brute Force Attacks, so I decided I will just revisit my SSH Server configuration, and change my SSH port to something else for the interim.
The dashboard that showed me the results at that point in time:
Then I decided I actually would like to set up an SSH Honeypot to listen...
5
Select One:
Encryption using PyCrypto in Python
While I'm learning a lot about encryption at the moment, I wanted to test out encryption with the PyCrypto module in Python using the Advanced Encryption Standard (AES) Symmetric Block Cipher.
Installing PyCrypto:
$ pip install pycrypto --user
PyCrypto Example:
Our AES Key needs to be either 16, 24 or 32 bytes long and our Init...
3
Select One:
Improve MySQL Write Performance Using Batch Writes
I am no DBA, but I got curious when I noticed sluggish write performance on a mysql database, and I remembered somewhere that you should always use batch writes over sequential writes. So I decided to test it out, using a python script and a mysql server.
What will we be doing
I wrote a python script that writes 100,000 records...
2
Select One:
Setup a Hugo Blog with the Kiera Theme
In this tutorial we will setup a Hugo Blog with the Kiera theme on Linux and will be using Ubuntu for this demonstration, but since Hugo runs on Go, you can run this on Windows, Linux or Mac.
Dependencies
We require git to download the theme from github, so first update your package managers indexes, and install git:
$ apt updat...
1
Select One:
Setup AWS S3 Cross Account Access
In this tutorial, I will demonstrate how to set up cross-account access for S3, from using two AWS Accounts.
Scenario
We will have 2 AWS Accounts:
a Green AWS Account which will host the IAM Users, this account will only be used for our IAM Accounts.
a Blue AWS Account which will be the account that hosts our AWS Resources,...
1
Select One:
Getting Started with Grafana Loki for Logging
Logging with Loki is AMAZING!
Background
In the past couple of months I've been working a lot with logging, but more specifically logging with Loki. As most of my metrics reside in Prometheus, I use Grafana quite extensively and logging was always the one that stood out a bit as I pushed my logs to Elasticsearch and consumed th...
1
Select One:
Using the libvirt provisioner with Terraform for KVM
In this post we will use the libvirt provisioner with Terraform to deploy a KVM Virtual Machine on a Remote KVM Host using SSH and use Ansible to deploy Nginx on our VM.
In my previous post I demonstrated how I provisioned my KVM Host and created a dedicated user for Terraform to authenticate to our KVM host to provision VMs.
O...
1
Select One:
Sending Slack Messages with Python
In this post I will demonstrate how to send messages to slack using python based on the status of an event.
We will keep it basic, that when something is down or up, it should send a slack message with the status, message, color and embed your grafana dashboard links inside the alert (or any links that you would like).
Create a...
2
Select One:
HTTPS for Local Development with MiniCA
In this tutorial we will use minica to enable us to run our web applications over HTTPS for local development.
To read more about about minica check out their website.
Generate Certificates
You can use their binary from their github page or use my docker image to generate the certificates to a ./certs directory:
$ docker run --u...
2
Select One:
Running Grafana Loki behind Nginx Reverse Proxy
In this tutorial I will demonstrate how to run Loki v2.0.0 behind a Nginx Reverse Proxy with basic http authentication enabled on Nginx and what to do to configure Nginx for websockets, which is required when you want to use tail in logcli via Nginx.
Assumptions
My environment consists of a AWS Application LoadBalancer with a Ho...
2
Select One:
Loading More Content
