Python

Search

connect spark to postgres; connect spark to database

spark = SparkSession.builder.config(conf=spark_conf).getOrCreate()

#make sure the necessary jar files are loaded
#@ opt/spark/jars folder

df = spark.read 
    .format("jdbc") 
    .option("url", "jdbc:postgresql://<host>:<port>/<db_name>") 
    .option("dbtable", "<schema>.<tablename>") 
    .option("user", "<username>") 
    .option("password", "<password>") 
    .option("driver", "org.postgresql.Driver") 
    .load()

Comment

PREVIOUS	NEXT

Code Example
Python :: pytorch optimizer change learning rate
Python :: how explode by using two columns pandas
Python :: python dict append
Python :: make the program title a name python
Python :: error handling in python using flask
Python :: .text python
Python :: pandas dataframe unique multiple columns
Python :: place legend on location matplotlib
Python :: Sorting Dataframes by Column Python Pandas
Python :: Export a Pandas dataframe as a table image
Python :: python for loop with increment
Python :: python dict remove key
Python :: making a basic network scanner using python
Python :: urllib.request.urlretrieve
Python :: return count of substring in a string
Python :: how to use turtle in python in python 3.9
Python :: calculate the distance between two points
Python :: apply same shuffle to two arrays numpy
Python :: extract tgz files in python
Python :: How to colour a specific cell in pandas dataframe
Python :: secondary y axis matplotlib
Python :: pandas reemplazar nan por cero
Python :: python print show special characters
Python :: end python program
Python :: flask port
Python :: install tensorflow gpu
Python :: django filter by date range
Python :: python collections Counter sort by key
Python :: push to pypi
Python :: tuple length in python

Search

PYTHON

connect spark to postgres; connect spark to database

ADD CONTENT