Uncategorized

Uncategorized

Pandora M1, a training monitor box (64 nodes x 8 GPUs support)

I started this project past year but never managed to finish due to many cumbersome issues tied to the mechanical part of the front panel which took me a lot of time, so this month I decided to finish it and this past weekend I was finally able to get some time to address the mechanical part. I’m very pleased with the results and there are still some things to finish but since it is starting to look great, I decided to share some photos and videos of the on-going project. This is the initial prototype so there is a unbearable amount of wiring around, however, I’m planning to hide all of this on a small case later and also design a PCB board for integrating all of this, which will help a lot with the wiring and clutter.

The main idea of this box is to provide a more tactile experience to monitoring distributed training GPUs, so I tried to make something that can fit on top of the desktop and had a more analog visualization with LED Bars and physical knobs that you can use. I iterated a lot on the design and I found this one to be quite elegant and also easy to operate.

Pandora M1

The main information that I wanted to put on the front panel was GPU Utilisation, GPU memory, GPU Temperature and also Power Usage. The tricky problem is that you want to visualize that for multiple nodes as well, so what I did was to use a LED Matrix to show each node so that when there is an issue on a node, it will blink that as well so you can go to that node and look at the stats as well. To select the node I added one big metal head on a potentiometer so that it is easier to move around on the nodes, it goes from 0 to 63, so it supports 64 nodes max, and given that each (at maximum 8) GPU has an individual LED Bar for it, the panel supports visualisation of 512 GPUs in total.

(more…)

by Christian S. Perone
Uncategorized

[pt-br] Dados das enchentes no Rio Grande do Sul (RS) em 2024

É muito triste ver as enchentes devastadoras que tem atingido o Rio Grande do Sul nos últimos anos. Decidi fazer este post pra tentar compreender melhor a escala e o impacto desses eventos usando algumas fotos de satélite e dados recentes sobre as enchentes. Maioria das imagens recentes são do MODIS (Moderate Resolution Imaging Spectroradiometer) do qual fiz um post em 2009 (artigo aqui) que usam 2 satélites (Terra e Aqua) para fazer cobertura quase diária em baixa resolução da terra inteira.

Estamos no meio de uma tragédia sem precedentes, por outro lado, este é um momento único para a coleta de dados por parte de pesquisadores e governo com a esperança de melhorar a modelagem desses processos hidrológicos para desenvolver sistemas de alerta e previsão de enchentes. Nunca antes observamos estes processos complexos nos rios do Rio Grande do Sul, este momento é extremamente importante para o futuro do RS.

PS: tentarei manter atualizado este post com novas imagens.

Avisos de licença das imagens:

Sentinel images: contains modified Copernicus Sentinel data 2024 processed by Sentinel Hub.
MODIS (Aqua/Terra): NASA/OB.DAAC.

(more…)

by Christian S. Perone
Uncategorized

Arduino WAN, Helium network and cryptographic co-processor

I was recently interested in the intersection of Machine Learning and RF and I was taking a look into LoRa modulation, which is based on Chirp Spread Spectrum (CSS), and ended up getting to know more about the Helium network. I still think that the most stupid piece of technology behind crypto mining is spending GPU/CPU/ASIC cycles to do proof-of-work (PoW), but in the Helium network, they did something quite interesting, which was to switch to something useful such as the proof-of-coverage instead of generating heat and burning energy. Therefore we can say that the miners are doing something useful by providing radio coverage, instead of purely generating heat.

(more…)

by Christian S. Perone
Uncategorized

Talk: Gradient-based optimization for Deep Learning

This weekend I gave a talk at the Machine Learning Porto Alegre Meetup about optimization methods for Deep Learning. In this material you will find an overview of first-order methods, second-order methods and some approximations of second-order methods as well about the natural gradient descent and approximations to it. I took some long nights to prepare this material, so I hope you like it! You can download the PDF of the slides by clicking on the top-right menu.

View Fullscreen
Skip to PDF content

– Christian S. Perone

by Christian S. Perone
Uncategorized

Visualizing sample simplex trajectories in Deep Learning

Softmax is a distribution over choices, it maps a vector into the probability simplex that is defined as \Delta_{n-1}=\{p\in\mathbb{R}^n\; \vert\; 1^\top p = 1 \; \; {\rm and} \;\; p \geq 0 \}, where the sum of all elements of the vector must equal 1. Softmax is used a lot in classification and I thought it would be interesting to visualize (when possible, on lower dimensions) the trajectories of individual samples in that simplex as predicted by the network while the network is being trained.

In the animations below you’ll see the trajectories of the sample individual sample (from the test set) over the simplex of 3 classes (dog, cat, horse) from CIFAR-10 and using a simple shallow CNN both with Adam and SGD. Each frame is generated after 10 optimization steps and the video is from 4 epochs with CIFAR-10 dataset with only the 3 aforementioned classes.

Trajectory of a CNN using Adam with LR of 0.001

Trajectory of a CNN using SGD with LR of 0.001 and momentum

by Christian S. Perone

Tags

announce arduino Article bayesian benford law c cnn convolutional neural networks covid deep learning electronics evolution evolutionary algorithms feature extraction ga genetic algorithm Genetic Algorithms genetic programming hardware Image Processing jit jython karl popper language model LLVM machine learning Math matplotlib modis News nlp Philosophy programming Pyevolve Python pytorch raspberry pi Science scikit.learn sigevolution tf-idf twitter ubigraph uncertainty visualization
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.