Cloud

Hello 👋, In this article we’re going to build a data pipeline that connects Kafka to MongoDB. In short, we’re going to add a MongoDB Sink connector to a Kafka Connect cluster and run a MongoDB instance in Docker to test the connector. By reading this article I hope that you will learn How to install the MongoDB connector in Kafka Connect How to configure the MongoDB connector How to create topics in Kafka using Confluent Tools How to monitor Kafka Connect using JConsole. Let’s get started! ...

Hello 👋, In this article I will give you some tips on how to improve the throughput of a message producer. I had to write a Golang based application which would consume messages from Apache Kafka and send them into a sink using HTTP JSON / HTTP Protocol Buffers. To see if my idea works, I started using a naïve approach in which I polled Kafka for messages and then send each message into the sink, one at a time. This worked, but it was slow. ...

Hi 👋👋 In this article I will explore the topic of sharding a Mongo Database that runs on Kubernetes. Before we get started, if you want to follow along, please install the tools listed in the prerequisites section, and if you want to learn more about sharding, check out this fantastic article Sharding Pattern. Prerequisites Kubernetes Cluster & kubectl mongosh helm Introduction Let’s install a MongoDB instance on the Kubernetes cluster using helm. ...

Introduction Hello 🙋‍♂️ In this article we will discuss a tool called Docker 🐬 Docker is a platform which allows to package individual applications in containers. This achieves application isolation at the OS level without the need to use virtualization technologies by making use of the OS APIs. Since it can be a little hard to get into Docker if you are new I will try to keep things short and concise. ...

Hi 🙋‍♂️, In this article I will talk about how to authenticate your applications to the Kubernetes API via the service accounts feature. Citing the Kubernetes docs, a service account for a pod: “provides an identity for processes that run in a Pod. When you (a human) access the cluster (for example, using kubectl), you are authenticated by the apiserver as a particular User Account (currently this is usually admin, unless your cluster administrator has customized your cluster). Processes in containers inside pods can also contact the apiserver. When they do, they are authenticated as a particular Service Account (for example, default).” 💡 ...

Hello 😄, In this article, I will demonstrate how to configure Kubernetes (minikube) to use OpenID Connect as an authentication strategy. We will cover the Resource Owner Password flow. Feel free chose the right authentication flow depending on your application’s needs. Please refer to this diagram in order to choose the flow: Note that the Client Credentials flow is not supported by Kubernetes. According to the official docs: “To identify the user, the authenticator uses the id_token (not the access_token) from the OAuth2 token response as a bearer token. “ ...

Hello! In this article we will discuss how to quickly get started with Kafka and Kafka Connect to grab all the commits from a Github repository. This is a practical tutorial which saves you some time browsing the Kafka’s documentation. Environment Kafka is bit difficult to setup, you will need Kafka, Zookeper and Kafka Connect at least. To simplify the setup we’re going to use Docker and docker-compose. Once you have Docker and docker-compose installed on your system, you can run a single command and get a Kafka environment for developing. ...

I’ve tried to use Google’s Firebase with the Firebase Admin SDK in my C# project. The good news is that the Admin SDK is easy to use and all you need to do is call two methods. The authentication part is handled by an environment variable which points to some .json file which you download. The bad news is that the SDK depends on gRPC and the dotnet version of it doesn’t work well on the Raspberry Pi. ...

Hello, I wanted to self host my WordPress blog and for that I’ve chosen a hosting provider which I no longer recommend.. Because I’m lazy and I don’t know how to configure web servers that well I’ve used XXX’s WordPress one click install, the package installed WordPress and did the following things: Enables the UFW firewall to allow only SSH (port 22, rate limited), HTTP (port 80), and HTTPS (port 443) access. Sets the MySQL root password, runs mysql_secure_installation, and creates a wordpress user with the necessary permissions. Sets up the debian-sys-maint user in MySQL so the system’s init scripts for MySQL will work without requiring the MySQL root user password. Creates the initial WordPress configuration file to set up salt keys and allow the WordPress instance to connect to the database. Disables XML-RPC to help prevent DDoS and other brute force attacks. (Should you require xmlrpc, run “a2disconf block-xmlrpc” from the terminal to disable blocking Modifies some of PHP’s settings to increase the maximum filesize and execution time. Enables the Apache rewrite module so the WordPress permalink feature will work. Configures Apache with UseCanonicalName On to mitigate CVE-2017-8295. This is pretty convenient as you can setup a fully functional blog in less than 10 minutes, the only thing that is missing is a email configuration. ...

Right after I’ve configured the FTP server, I wanted to be able to “name” my server and so I’ve installed a lightweight DNS server. (context: previous article) “dnsmasq is a lightweight DNS, TFTP, PXE, router advertisement and DHCP server. It is intended to provide coupled DNS and DHCP service to a LAN.” I’ve chosen this over the bind 9 dns server because it was very easy to setup, I almost got it right the first time. ...