Marcel Castro

What KL Divergence has to do with Large Language Model

Regularization, LLM, transformers, DPO, SFT, RLHF

KL Divergence and its role on Large Language Model Training.

AI Agents Revolution

LLM, Agents

High level overview of AI Agents and evolutions.

Thinking Tokens

LLM, transformers, Tokens

Concept of thinking tokens to improve model performance while reasoning.

Overview of LLM Optimizations Techniques

machine_learning, transformers, LLM

Overview of current Large Language Model Optimization techniques for both inference and training

Understanding Mixture of Experts

sagemaker

transformers

NLP

MoE

Details on Mixture of Experts and how to run it.

Flash Attention - Fast and Memory Efficient Attention Mechanism

The attention layer is the main bottleneck in scaling longer sequences in LLM (Large Language Models), as its runtime and memory increase quadratically in the sequence…

How Attention Mechanism works in Transformers

NLP

transfomers

PyTorch

Simple implementation of self-attention using PyTorch.

Text Moderation - Toxicity Classification using Amazon Comprehend API

This notebook will capture different methods of performing text moderation, in special with focus on toxicity classification. It will be divided into 3 parts:

Train Falcon with near-linear scaling using Sharded Data Parallelism technique in SageMaker Model Parallelism Library

This notebook’s CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.

Chat Conversation with Falcon 7B Instruct using Amazon SageMaker JumpStart

This notebook has the objective to test the Langchain chat conversation using the Falcon 7B Instruct LLM model using SageMaker JumpStart.

ChatGPT over your data

Notebook inspired by: - Tutorial: ChatGPT Over Your Data - Build a GitHub Support Bot with GPT3, LangChain, and Python - Meet Bricky - a conversational bot using OpenAI

Prompt Engineering 101

Wikipedia Definition: Prompt engineering is a concept in artificial intelligence (AI), particularly natural language processing (NLP). In prompt engineering, the description…

Visual ChatGPT - Talking, Drawing and Editing with Visual Foundation Models

Visual ChatGPT connects ChatGPT and a series of Visual Foundation Models to enable sending and receiving images during chatting.

Semantic Search with Pinecone and OpenAI

This repository uses OpenAI Embedding API to generate language embeddings and store them in pinecone. Ref. ChatGPT Retrieval Plugin.

Stable Diffusion on Amazon SageMaker

NLP

transfomers

stablediffusion

Journey to learn about transformer models in SageMaker. Inspired by philschmid.

Notes on Natural Langage Processing with Deep Learning

NLP

word2vector

A notebook that reproduces some of the teaching content from Stanford CS224n Natural Language Processing with Deep Learning

Joint Image & Text Embeddings

First we need to install sentence-transformers

Multilingual Joint Image & Text Embeddings

%%capture
!pip install sentence-transformers

Machine Learning Model Monitoring

ML

data drift

model drift

MLOps

Notes on machine learning model monitoring concepts, challenges and howto.

Github Actions and Snowflake Integration

github

snowflake

CICD

Using github actions to create a CICD data pipeline in snowflake data.

Dimensionality Reduction - Non-negative Matrix Factorization - NMF

jupyter

NMF

Dimensionality

TF-IDF

PMI

A notebook to evaluate topics on non-conformance reports.

Generative Deep Learning

machine_learning

This repository is intended as place to keep my current experimentations on generative deep learning using tensorflow.

Principal Component Analysis - PCA

jupyter

Dimensionality

PCA

Step-by-step use of PCA for dimensionality reduction.

Using Keras Tuner for hyperparameter tunning

kerastuner

tensorflow

Adaptation from notebook from C3_W1_Lab_1_Keras_Tuner from DeepLearningAi MLOPs Specialization - Course 3

Feature Engineering with Tensorflow

jupyter

tensorflow

Notebook from C2W1_Assignment lab assignment from DeepLearningAi MLOPs Specialization - Course 2 -

My Title for Template

jupyter

Dimensionality

PCA

template

This is a template description.