Sometimes you want to query your DWH and put the data in higher level storage for example Mongo see the following python code that can...

You don't need to hardcode your credentials in your program just read them - use the following code, read access and secret and select...

The following code will run os command every 10 second in parallel to your code, the need was raised when using aws with token and need...

Add the following lambda to handle the stream data, and this will add new line '\n' to each line: from __future__ import print_function...

You have a presto and you want to connect Python to it .... from pyhive import presto cursor = presto.connect(host='localhost',...

Michael Yin - wrote a very nice snippet of code for ipython and Jupiter notebook from IPython.core.magic import register_line_magic...

You want a oneliner python code: res = dict((v,k) for k,v in orig.items()) source: https://stackoverflow.com/questions/1031851/how-do-i-e...

So I would like to check a value of field in parquest file for a row. or I would like to get a parquet to csv file. Use Python of course:...

You can use SQL including window functions on Spark dataframe nice article on this can be found here : https://medium.com/jbennetcodes/ho...

How can you calculate the number of hours within the night shift (for example between 18-05) or during weekends (Saturday/Sunday)? I...

Look on the following Oracle article - how to create groups for sequence records using SQL window functions : select val ,row_number()...

So.... We'll try to give useful ideas, tips and peace of codes for you (and us) to explore and use. in this case the quickest way to get...