If you have a case of CDC - change data capture, in pandas and you want to have the last and most updated row per entity, CDC means that...

This is an ongoing post with tips and tricks to use with Jupyter and Pandas - sure there are many more send me and I'll add them...

To install use docker-compose (how to it install on ec2 see the previous post): https://airflow.apache.org/docs/apache-airflow/stable/sta...

Well just spent a couple of hours figuring it out... follow this link and some common sense and you'll get there...

If you write python code in AWS environment you should consider use awswrangler. I wrote a simple example - for finding a dataframe...

There are many ways how to know your data prior to manipulate and process it: you can use if you have the dat in sql you can use sql to...

You can run sql and than visualise using pandas and plot.... and it's sooo easy : !pip install ipython-sql import sqlalchemy %load_ext...

So I've spent (too much time and got to much errors in the way) what worked for me in the end is: mkdir mysql create docker-compose.yml...

So you have a file like json but with single quote and you want to load it to mongodb or just to have a valid Json .... import ast,json...

Sometimes you get a very long PDF and in the end, you want only a couple of pages out of it. us the following code to grab these pages :...

There are some times where you want to add to grouped by datagram additional columns, for example, if you group by substring of the...

Well sometimes it just works when having the s3 as the destination but the following worked even when the above didn't: from io import...

There may be a situation when you have a picture in your clipboard and you want to save it in a file. Or you are working with Pandas and...

If you have a very large file on S3 or on the web use smart_open (or even if you have local file gzip..) It is a Python 3 library for...

So you have a dataframe and you want to group by and then take only some of the groups or take only groups other then other. After group...

You have a gzip (or other) file on S3 and you want it in dataframe: import pandas as pd import boto3 df = pd.read_csv('s3://bucket/file.c...

In this short post, I've collected useful links for EC2 improvements such as how to configure the local discs, how to add UI if you need...

Well, you want to send a small snap of a movie - let's say in Whatsapp - for example, part of Life of Brian - because you want to prove a...

So you have You have an expense report, for example, coming from aws CUR (https://aws.amazon.com/premiumsupport/knowledge-center/cost-us...

Sometimes you need to access your aws services with specific profile (or not default) The easiest way to do so : import boto3 new_session...