Python versus ETL tools : r/dataengineering - Reddit
Python is a language, trying to do data engineering using open ("file.parquet", "r") or even better using boto3 to read individual objects would be outrageously complex and idiotic. It's the tools/libraries you …
ETL using pure python (no Pandas) : r/dataengineering - Reddit
Jan 23, 2023 · I’ve heard that using pure python is a good step before using Spark, but I can’t find an example of ETL using just pure python. How would that work? Is there an example script for an ETL …
Python over SSIS for ETL : r/BusinessIntelligence - Reddit
Oct 18, 2023 · I keep seeing this craze of using Python to ingest data and also for ETL over something like SSIS. To me, SSIS is hugely performant and can be done by non programming people. What am …
Python and ETL : r/dataengineering - Reddit
Nov 4, 2021 · Python is rarely a good choice for ETL. With modern data warehouses like Snowflake, you can write a few lines of SQL and leave the query optimizer do their work: you don't have to worry …
Is SSIS still the preferred choice for ETL? : r/SQLServer - Reddit
Jun 10, 2022 · I used SSIS for a number of years in the 2005 - 2015 timeframe, but haven't touched it since. Is this still the preferred tool for ETL in the Microsoft space? Outside of the MS world, it seems …
Where to start with Python for data pipelines? - Reddit
Aug 26, 2023 · Without knowing the specifics about your data sources, I'm inclined to say that you can build an ETL workflow using Python (pandas is your friend). With that said, I'm a fan of the Microsoft …
The expert way of structuring a project for Python ETL. : r/ETL
Aug 29, 2019 · The expert way of structuring a project for Python ETL. Hi, I have recently moved from Informatica based ETL project to Python/Pyspark based ETL. I want to know the best way to …
How can I schedule python ETL code? : r/dataengineering - Reddit
Jun 29, 2023 · But I feel like if you're looking for something python based with a nice UI, check out Dagster or Prefect. Dagster bills itself as more data ETL oriented, while prefect seems to be more …
Best place to learn Python for ETL? : r/learnpython - Reddit
Dec 22, 2022 · Python is versatile enough to ETL a lot of ways with a lot of database types. I would focus on learning about ETL itself then circle back to python. A lot of packages are out there to help …
ETL with Python and SQL - Keep it all in one file or separate ... - Reddit
Nov 17, 2022 · This might be the dumbest question you read all day, but for those of you who use Python for ETL, are the SQL queries incorporated into the Python file where the transformation is …