Apache Flink Checkpoints on S3 and S3 compatible storage

Hello, Recently someone working at Yahoo emailed me regarding an old thread I’ve started on the Apache Flink user mailing list. I’ve replied to the e-mail but also decided to turn the reply into a blog post, because it might help other people as well. Email Hi, I was able to get it working after tinkering with it. The issue was mainly a miscommunication, we didn’t formally know which authentication method we were using in AWS....

October 28, 2022 · 2 min · Denis Nuțiu

Using confluent-kafka-go on MacOS M1

Hello, TLDR; brew install librdkafka openssl@3 pkg-config export PKG_CONFIG_PATH="/opt/homebrew/opt/openssl@3/lib/pkgconfig" go test -tags dynamic ./... I’ve been transition from a Linux machine to a MacOS M1 machine at work and when I ran tests for a Golang project, I noticed that the test failed on modules depending on librdkafka. Initially I’ve had problems with Kafka on MacOS M1 on Docker, since I was using an older image version that didn’t have any arm64 build, updating to images to version 7....

October 17, 2022 · 2 min · Denis Nuțiu

How to identify similar images using hashing and Python

Hi 👋, In this article I would like to talk about image hashing. Image hashing algorithms are specialized hashing functions that output the hash of an image based on the image’s properties. Duplicate images output the same hash value and visually identical images output a hash value that is slightly different. To simplify hash("white_cat") = "aaaa" hash("brown_cat") = "aaba" hash("car") = "xkjwe" Some use cases for image hashing are: Duplicate Image Detection Anti-Impersonation / Image Stealing Image filtering Reverse image search Let’s play around with image hashing techniques using Python and the ImageHash library....

September 21, 2022 · 4 min · Denis Nuțiu

DeMorgan’s Law

DeMorgan’s law is a simple law that I learned at UPT during one of my hardware classes. While it is useful in hardware it, it is also useful when writing programs. If you have a condition like not (A and B), you can rewrite it to !A or !B. if __name__ == '__main__': a = True b = True if not (a and b): print("True") else: print("False") if not a or not b: print("True") else: print("False")

August 25, 2022 · 1 min · Denis Nuțiu

Object Pool Pattern

Hi 👋 In this article we’ll talk about the Object Pool pattern in Golang. The Object Pool pattern is a design pattern used in situations when constructing objects is a costly operation, for example building an HTTPClient or DatabaseClient object can take some time. By having a pool of resources, the resources are requested from the pool when needed and then returned when not needed so they can be reused later....

August 21, 2022 · 5 min · Denis Nuțiu

HTTPClients are reusable

Hi 👋, I wanted to write this article to tell you that instantiating a HttpClient class is often a costly operation. If you have a method that does something like: public class Main { static void myMethod() { HttpClient client = new HttpClient() // do stuff } } You should refactor it ASAP. Either move the HttpClient into the class and perhaps ensure that the class is a Singleton or retrieve the HttpClient instance using an Object Pool pattern like the HttpClient class from System....

June 23, 2022 · 1 min · Denis Nuțiu

Go Pattern: Sorting a slice on multiple keys

Hi 👋 In this article I want to highlight a simple pattern for sorting a slice in Go on multiple keys. Given the following structure, let’s say we want to sort it in ascending order after Version, Generation and Time. type TheStruct struct { Generation int Time int Version int } The way we sort slices in Go is by using the sort interface or one of the sort.Slice functions. To sort the slice after the above criteria we’ll call slice....

May 12, 2022 · 2 min · Denis Nuțiu

How I got my PR merged into Apache Flink

Hi 👋 This is a short story on how I got my pull request merged into Apache Flink. It started with the need to set CPU and Memory limits to Flink jobs running under Kubernetes. The first thing I did was to join the user mailing list and ask around if someone has encountered the issue and if there’s a solution to it. The people from the mailing list were very friendly and they pointed me to an existing ticket on the Flink jira board, which was exactly what I needed....

February 13, 2022 · 2 min · Denis Nuțiu

How to install a specific Python version on Linux

Hello, 👋 In this article I will show you how to install Python versions on Linux using the following methods: compiling from source, dead snakes ppa and pyenv. To make things easier, if you want to follow along in an environment that you can break, you can create a local Kubernetes cluster using Minikube. Next, I’m going to use the following yaml file to create an Ubuntu pod: apiVersion: v1 kind: Pod metadata: name: ubuntu labels: app: ubuntu spec: containers: - image: ubuntu command: - "sleep" - "604800" imagePullPolicy: IfNotPresent name: ubuntu restartPolicy: Always Save the above yaml in a file ubuntu_pod....

February 5, 2022 · 2 min · Denis Nuțiu

Pytest Fixtures and Yield

Hi 👋 In this short article I want to explain the use of the yield keyword in pytest fixtures. What is pytest? Pytest is a complex python framework used for writing tests. It has lots of advanced features and it supports plugins. Many projects prefer pytest in addition to Python’s unitttest library. What is a fixture? A test fixture is a piece of code that fixes some common functionality that is required for writing the unit tests....

December 14, 2021 · 3 min · Denis Nuțiu