Jardim on rfbatista

Agents Memory

Wed, 07 Oct 2026 00:00:00 +0000

References

https://thilo-hermann.medium.com/stop-mixing-cdm-ontology-and-knowledge-graphs-0d1d154e0c31

API Engineering

Wed, 07 Oct 2026 00:00:00 +0000

References

https://medium.com/@gaddamnaveen192/designing-a-notification-system-that-can-handle-10-million-users-cad6b4970b78 https://medium.com/javarevisited/how-i-design-apis-as-senior-developer-for-better-results-7209d40e6a29

Algorithms

Sat, 13 Sep 2025 00:00:00 +0000

References

https://github.com/imteekay/algorithms

AWS CDK

Fri, 12 Sep 2025 00:00:00 +0000

npm install -g aws-cdk
In a empty folder run: cdk init app --language python
pip install -r requirements.txt

Actor Model

Sun, 24 Aug 2025 00:00:00 +0000

Registry

Node

Actor

References

10 Lessons From Implementing The Actor Model • Vagif Abilov • GOTO 2023

Actor Model

Sun, 24 Aug 2025 00:00:00 +0000

https://github.com/gorilla/mux https://github.com/hashicorp/go-metrics https://github.com/asaskevich/govalidator

CLI

https://github.com/briandowns/spinner

Tests

https://github.com/onsi/ginkgo

Web Cookies

Sat, 25 Jan 2025 00:00:00 +0000

References

https://stackoverflow.com/questions/73547776/how-to-redirect-from-one-domain-to-another-and-set-cookies-or-headers-for-the-ot/73599289#73599289

Container

Mon, 20 Jan 2025 00:00:00 +0000

Container runtime

Containers are implemented using Linux Namespaces and Cgroups

Sysbox

Similar to VM in its capabilities, being possible t

Cgroups

Provide a way to limit the amount of resources, such as CPU and memory

Namespaces

Virtualizes resources like file systems or networking

References

Container Runtimes Part 1: An Introduction to Container Runtimes

API Managment

Tue, 07 Jan 2025 00:00:00 +0000

Load balancing Azure OpenAI

Normal scenario:

token and model quota is available
api return 200 Throttled scenario:
quota limits is applied
api managment retry to a different resource

References

https://learn.microsoft.com/en-us/azure/developer/python/get-started-app-chat-scaling-with-azure-api-management?tabs=github-codespaces%2Cinitial-deployment

CosmosDB

Tue, 07 Jan 2025 00:00:00 +0000

Whats is NoSQL

distributed
schemafree

There is a local emulator w

Primary key and secondary key are used to rotate access keys without loosing connection access

Container

Documents

zip_code

Management

Azure portal
Azure CLI
PowerShell
ARM Templates
Bicep Templates
Rest API

Automatic indexing

The hierarchy of a json tree, is flatened as a list of keys and values:

/locations/0/contry: Germany
/locations/1/contry: Berlin
/exports: Examples

This structure is automatically indexed. But you can finetune with a custom policy

C++: artigos

Thu, 02 Jan 2025 00:00:00 +0000

Artigos

Filesystem

https://www.cppstories.com/2024/common-filesystem-cpp20/#1-creating-directories

Desired resilience properties

Thu, 02 Jan 2025 00:00:00 +0000

Desired resilience properties

Fault isolation
Sufficient capacity
Timely output
Correct output
Redundancy

Categories of failure

Single point of failure
Excessive load
Excessive latency
Misconfiguration bugs
Automate as much as possible

CAOS Engineering

Fault injection simulator

Networking Basics

Thu, 02 Jan 2025 00:00:00 +0000

Two or more computers connected together DNS (Domain Name System)

Types of netwok

Topologies

Ring
Point-to-point
Mesh
Star
BUS
Tree
HYbrid

Building blocks

Devices (MAC Address)
Connections - type of connection Ex.: wireless
Switches - connect various devices
Routers
Network servers

OSI Model

Network Services

DNS, email, internet chat, printer sharing…

IP Addresses

IPv4
IPv6
NAT public and private
- network address translation
- private LAN
- public WAN

IPv4

192.168.2.14:80

192.168.2 = network id
.14 = host id
80 = port

IPv6

Written in hexadecimal

Airflow

Wed, 20 Nov 2024 00:00:00 +0000

Airflow Components

Scheduler: keep track of dags
Workers:

Executors:

Local
Celery <– always on
Kubernetes <– on demand
Celery + Kubernetes

Database

pgboucer vs max_connections

Pipeline design:

workflows
tasks
inputs
outputs

Local development:

spend time setting up your locals
try to debug in local

Infrastructure

Kubernetes

use iac

ci/cd pipeline

scheduler (mount volume)
scheduler pull github
scheduler docker image

Logging

Monitoring and alerting

monitor resources and events
prometheus and grafana to monitor
airflow stats

Resource allocation

Containers from scratch

Sat, 12 Oct 2024 00:00:00 +0000

namespaces: kernel groups accessed by specific processes. There are different kind of namespaces
capabilities: set limits to uid 0
cgroups: mechanism to limit usage of memory, disk, cpu-time and other resources.
setrlimit: similar mechanism to cgroups but is older and can do somethings cgroups cant.

need to isntall sudo apt-get install libseccomp-dev

References

Modules in C

Sat, 12 Oct 2024 00:00:00 +0000

There are not formal modules in the C language
Header file contains function and type declarations
Source file (.c) contains concrete implementations
Header file provides the interface to a module

SQLite

Sun, 06 Oct 2024 00:00:00 +0000

SQLIte is faster than filesystem

PRAGMA journal_mode = WAL;
PRAGMA busy_timeout = 5000;
PRAGMA synchronous = NORMAL;
PRAGMA cache_size = 1000000000;
PRAGMA foreign_keys = true;
PRAGMA temp_store = memory;

PRAGMA journal_mode = WAL; write and reader don’t block

PRAGMA synchronous = NORMAL;

PRAGMA busy_timeout = 5000; avoid SQLITE_BUSY errors to happen

Use IMMEDIATE transactions so sqlite respest the busy_timeout. It can be configure at pool level in GO mydb.db?_txlock=immediate

ZIG Guide

Fri, 26 Jul 2024 00:00:00 +0000

Variaveis são snake_case e fuções são camelCase

Assignment

(const|var) identifier[: type] = value

const - indica que o identifier é uma constante que armazena um valor imutável
var - indica que o identifier é uma variavel que armazena um valor mutável
type - é uma anotação do tipo do identifier que pode ser omitido se o value pode ser inferido

const constant: i32 = 5; // signed 32-bit constant
var variable: u32 = 5000; // unsigned 32-bit variable

// @as performs an explicit type coercion
const inferred_constant = @as(i32, 5);
var inferred_variable = @as(u32, 5000);

Array

Array literals podem ser omitidos com o uso de _:

Ways to level up your communication

Tue, 19 Mar 2024 00:00:00 +0000

1. Organize your ideas beforehand

Categorize knowledge that are important for that discussion, so it is easier to retrieve information about a specific topic and associated facts

Tools:

Mindmap

2. Master BLUF (Bottom line up front)

State the core message first, and follow up with necessary context. That way you reduce friction.

3. Storytelling

Give structure for the context detail. Storytelling is best suited for complex journey, justify a timeline or make an abstract concept more relatable.

Architecture Decision Record

Wed, 21 Feb 2024 00:00:00 +0000

(https://en.wikipedia.org/wiki/A_Pattern_Language)[https://en.wikipedia.org/wiki/A_Pattern_Language] (https://microservices.io/post/misc/2022/01/26/the-microservices-pattern-language.html)[https://microservices.io/post/misc/2022/01/26/the-microservices-pattern-language.html] (https://twitter.com/crichardson/status/1661757842477465608)[https://twitter.com/crichardson/status/1661757842477465608]

The Humble Programmer

Wed, 21 Feb 2024 00:00:00 +0000

intellectually manageable programs
subset of intellectually manageable programs reduce the solution space
proof of correctness over tests, build on top of a programs designed to be correct and no the other way around
vital abstraction patterns
avoid to write smart programs

https://www.cs.utexas.edu/~EWD/transcriptions/EWD03xx/EWD340.html

Tutoring systems

Sun, 18 Feb 2024 00:00:00 +0000

Design and development methods:

needs assesment
cognitive task analysis - how domain experts think when solving problems
initial tutor implementation
evaluation

An intelligent tutor system should enable the student to work to the successful conclusion of problem solving.

Represent student competence as a production set.
Communicate the goal structure underlying the problem solving.
Provide instruction in the problem solving context.
Promote an abstract understanding of the problem-solving knowledge.
Minimize working memory load.
Provide immediate feedback on errors.
Adjust the grain size of instruction with learning.
Facilitate successive approximations to the target skill

4 Components architecture:

Vector Databases

Sun, 18 Feb 2024 00:00:00 +0000

References

https://www.youtube.com/watch?v=5hAGhmHNTQQ

Database from scratch

Tue, 06 Feb 2024 00:00:00 +0000

Persistence: como não perder dados
Indexing: como consultar dados
Concurrency: como lidar com multiplos clientes

Persistence

Indexing

Existem dois tipos de queries:

Analytical (OLAP): Geralmente envolvem um grande numéro de dados, com agregação, agrupamento, or joins.
Transactional (OLTP): consultas que envolvem apenas uma pequena parte dos dados.

Transactional (OLTP)

Azure OpenAI

Sat, 03 Feb 2024 00:00:00 +0000

Puorpose of quotas

To regulate the number of requests, the volume of texts tokens, or the amount of computational power utilized by OpenAI API user

Quota Allocation

Subscription receive a quota allocation, determined by geographic region and regions quantified by token-per-minute (TPM)

Rate limit

tokens per minute (TPM)
request per minute (RPM)

Conversion metric

The RPM ratio is set to 6 RPM for every 1000 TPM available

Azure API Management (APIM)

Azure Monitor and Log Analytics

Azure OpenAI Monitoring Metrics

Azure OpenAI requests
Processed prompt tokes
Generated completion tokens
Processed inference tokens
Processed fine tuned training hours

References

Learn Live: Monitoring Azure OpenAI Monitor OpenAI Monitor OpenAI

Database design

Sat, 03 Feb 2024 00:00:00 +0000

(https://cs-people.bu.edu/mathan/publications/fnt23-athanassoulis.pdf?utm_source=hackernewsletter&utm_medium=email&utm_term=books)[https://cs-people.bu.edu/mathan/publications/fnt23-athanassoulis.pdf?utm_source=hackernewsletter&utm_medium=email&utm_term=books] (https://www.postgresql.org/docs/current/query-path.html)[https://www.postgresql.org/docs/current/query-path.html] (https://ocw.mit.edu/courses/6-830-database-systems-fall-2010/)[https://ocw.mit.edu/courses/6-830-database-systems-fall-2010/] (https://blog.marcua.net/2009/06/03/mit-database-systems-6830-ta-course-notes.html)[https://blog.marcua.net/2009/06/03/mit-database-systems-6830-ta-course-notes.html]

Dependency injection system for Go

Sat, 03 Feb 2024 00:00:00 +0000

Parameter Object

São objetos com o propósito de carregar para métodos ou funções como as funções New* que criam objetos.

type ClientParams struct {
 fx.In

 Config ClientConfig
 HTTPClient *http.Client
}

func NewClient(p ClientParams) (*Client, error) {
 return &Client{
 url: p.Config.URL,
 http: p.HTTPClient,
 // ...
 }, nil

Result Object

Similar ao Parameter Object é utilizado para carregar os resultados de uma função ou método, o objeto é definido exclusivamente por uma função ou método

Frontend Development

Sat, 03 Feb 2024 00:00:00 +0000

Rendering patterns

Static rendering - HTML are rendenred at build time, that way you can cache in CDN
ISR
SSR
CSR

Core Web Vitals

TTFB
FCP
TTI
FID
CLS
LCP

Architecture

Application architecture:

component model
rendering and loading
routing and transitions
data/state management

Rendering and loading

Bigger javascript bundle will always means a slower performance Serverside rendering have a faster FTP

SSR:

Performance
SEO
Data fetching

Tecniques:

SSR
SSR with hydration

NeuralSearchX: Serving a Multi-billion-parameter Reranker for Multilingual Metasearch at a Low Cost

Sat, 03 Feb 2024 00:00:00 +0000

Multistage ranking

Our solution

A multi-stage metasearcgh engine, stage:

candidade retrieval step using federated search over web search engines along with sparse and dense retrieval from private collection
reranking step based on previously selected documents with a neural model, score based on relavancy to a query
the best documents are sent to a highlighter model, that select s the most relevant sentecences from each document

1st stage candidates retrieval

Use sparse and dense retrieval methods (sparce to find exact matches and dense handles semantical meaning) Before indexing we split the contents of long documents over multiple windows windows of 150 words with stride of 75 words

Procedural Generation

Sat, 03 Feb 2024 00:00:00 +0000

Procedural Generation

Evolutionay Computation

É uma categoria de algoritmos baseados na idéia de uma população baseada em individuos. Em cada geração (iteração do algoritmos) os individuos são atualizados Isso acontece em duas operações mutação e seleção

mutação: age em cada individuo aplicando pequenas modificações
seleção: age em toda população removendo o pior individuo após a aplicação de uma fitness function

A fitness function utilizada para avaliar as soluções e determinar a condição de término do algoritmo. Esse método funciona bem em aplicações com grande numero de possiveis soluções.

Sistemas distribuidos

Sat, 03 Feb 2024 00:00:00 +0000

Fallacies of Distributed Computing

The network is reliable
Latency is zero
Bandwidth is infinite
The network is secure
Topology doesn’t change
There is one administrator
Transport cost is zero
The network is homogeneous

(https://dancres.github.io/Pages/)[https://dancres.github.io/Pages/] (https://ferd.ca/a-distributed-systems-reading-list.html?utm_source=hackernewsletter&utm_medium=email&utm_term=code)[https://ferd.ca/a-distributed-systems-reading-list.html?utm_source=hackernewsletter&utm_medium=email&utm_term=code] (https://github.com/aphyr/distsys-class)[https://github.com/aphyr/distsys-class]

AWS ECS

Sat, 06 Jan 2024 00:00:00 +0000

Container registry

Amazon ECR privaty registry

Cluster

Task definition

ECS Cluster have tasks definitions. Taks definitions specifies parameters for application including which containers to run

WebRTC

Thu, 02 Nov 2023 00:00:00 +0000

With WebRTC you can stream audio, video (or both) and arbritary data between browsers performing peer-to-peer communication. This is really powerful. In this article I will explain how WebRTC

acquiring audio or video
communicating audio and video
communicating arbritary data

Javascript API

MediaStream

Represents a stream of audio and/or video. It allows access to camera and microphone from host device. To get a MediaStream you can call navigator.getUserMedia

var constraints = { video: true };

function onSuccess(stream) {
 var video = document.querySelector("video");
 video.src = window.URL.createObjectURL(stream);
}

function onError(error) {
 console.log("failed", error);
}

navigator.getUserMedia(constraints, onSuccess, onError);

With constraints you can control the content of MediaStream, like frame rate, type, width and height of the video. To use audiowe have a similar api.

IO Pipes

Sun, 15 Oct 2023 00:00:00 +0000

With io.Pipe we have a convenient way to connect a writer to a reader, really similar to how channels work

pr, pw := io.Pipe()

References

justforfunc #19: mastering io.Pipes

PyTorch

Sun, 15 Oct 2023 00:00:00 +0000

How models work in PyTorch

nn.Module - fornece uma base para a criação de modelos personalizados, junto com funcionalidades que ajudam no treinamento.

Duas funções precisam ser definidas:

__init__ definimos as várias camadas que serão utilizadas
forward definem a sequencias em que as camdas serão utilizadas

Camadas no PyTorch

nn.Conv2d the convolutional layer
nn.BatchNorm2d normalização da saida da camada de covolução
nn.ReLU função de ativação
nn.MaxPool2d aplica max pooling na saída com o tamanho do kernel
nn.Dropout aplica dropout na saida com uma dada probabilidade
nn.Linear camada totalmente conectada
nn.Sequential não é tecnicamente uma camada mas ajuda na combinação de diferentes operações que são parte da mesma etapa

Treinamento

ResNet

Sun, 15 Oct 2023 00:00:00 +0000

Redes Neurais Residuais (ResNet)

Com o aumento de camadas nas redes neurais os gradientes da função de perda acabam tendendo 0 dessa formas os pesoas da rede não são atualizados (conhecido como Vanishing Gradient Problem).

Referencias

[1] https://arxiv.org/pdf/1512.03385v1.pdf [2] https://medium.com/@ibtedaazeem/understanding-resnet-architecture-a-deep-dive-into-residual-neural-network-2c792e6537a9 [3] https://blog.paperspace.com/writing-resnet-from-scratch-in-pytorch/

GO - Strings

Mon, 11 Sep 2023 00:00:00 +0000

Uma string é uma sequencia imutavel de bytes. As strings são convencionalmente interpretadas como sequencias de pontos de código Unicode (runas) codificadas em UTF-8.

A função lendevolve o número de bytes não de runas

s :+ "hello, world"
fmt.Println(len(s)) // 12
fmt.Println(s[0], s[7]) // "104 119"

O iésimo byte de uma string não corresponde diretamente ao iésimo caracter, pois a codificação UTF-8 de um ponto de código que não seja ASCII exige mais.

Gorrotinas e Canais

Sat, 15 Jul 2023 00:00:00 +0000

go f() // cria uma nova gorrotina que chama f()

Canais

Os canais sãos os meios de comunicação entre as gorrotinas Cada canal é um comunicador de um tipo particular

ch := make(chan int) // ch é do tipo chan int

Um canal é uma referência a estrutura criada por make

instrução de envio: ch <-x
instrução de recpção em uma instrução de atribuição: x = <-ch
instrução de recepção com resultado descartado: <-ch

Para fechar um canal

close(ch)

GO - Tipos compostos

Wed, 19 Apr 2023 00:00:00 +0000

Array

Um array é uma sequencia de tamanho fixo de zero ou mais elements

var q [3]int = [3]int{1,2,3}
q := [...]int{1,2,3} // com a reticencia o tamano será determinado pela quantidade de inicializadores

O tamanho de um array faz parte do seu tipo

q := [3]int{1,2,3}
q = [4]int{1,2,3,4} // erro de compilação: não é permitido atribuir [4]int a [3]int

Arrays são inflexiveis

Fatias (slices)

Uma fatia representa um sequencia de tamanho variavel cujo elementos tem o mesmo tipo. Uma fatia tem tres elementos:

Links que guardei

Mon, 01 Jan 0001 00:00:00 +0000

Blogs and Personal Website

Software Architecture

27 Fundamental Techniques for Software Architects

Articles

Frontend

devops

Building Hashicorp Nomad Cluster With Raspberry Pi. Part 1

Music

https://www.youtube.com/@spiritual_brother

Youtube Channels

Real-time applications

Building Real-time Apps with Go | Azim Pulat

Talks

Scaling Yourself - https://www.youtube.com/watch?v=FS1mnISoG7U Lone Star Ruby Conference 2010 Real Software Engineering by Glenn Vanderburg

LLM Inference

Mon, 01 Jan 0001 00:00:00 +0000

The lifecycle of a model is implemented in the following steps:

Open the model file and read the shapes of the weights, but don’t load the weights yet.
Using the loaded shapes and optional metadata, instantiate a model struct with Tensors, representing the shape and layout of each layer of the NN.
Compile the model struct and it’s forward function into an accelerator specific executable. The forward function describes the mathematical operations corresponding to the model inference.

Para ler

Mon, 01 Jan 0001 00:00:00 +0000

O que é uma antibiblioteca? De forma simples, uma antibiblioteca é uma coleção particular de livros ainda não lidos. Em vez de celebrar tudo o que você já sabe, ela é uma homenagem a tudo o que você quer explorar.

◌Tiny Experiments: How to Live Freely in a Goal-Obsessed WorldAnne-Laure Le Cunff
◌The Penguin Modern Classics BookHenry Eliot
◌The Human Animal: A Personal View of the Human SpeciesDesmond Morris
◌Reflexos do mundo: Trabalhar e viverFabien Toulmé
◌The Making of Prince of Persia: Journals 1985-1993Jordan Mechner
◌A cidade e suas muralhas incertasHaruki Murakami
◌Building Scientific Apparatus: A Practical Guide to Design and ConstructionJohn H. Moore et al.
◌The Origins of EfficiencyBrian Potter
◌Maintenance of Everything: Part OneStewart Brand