How to export dataFrame csv/excel file to a specific directory
up vote
0
down vote
favorite
Databricks dataFrame
I have a dataFrame that I would like to export the result to a specific path/directory.
By doing display(df), you got a button that you can press that will give you a download off the result that you can open in excel.
How can I do this by using python? I am trying to make a job in databricks that run in a specific schedule, and I want it to run my entire notebook, and in the end make it export the final result and save it in a specific directory so I can go back in time and see some old result.
python csv dataframe pyspark databricks
edited Nov 9 at 9:56
Mayank Porwal
3,7181621
asked Nov 9 at 9:50
celllaa95
187
add a comment |
up vote
0
down vote
favorite
Databricks dataFrame
I have a dataFrame that I would like to export the result to a specific path/directory.
By doing display(df), you got a button that you can press that will give you a download off the result that you can open in excel.
How can I do this by using python? I am trying to make a job in databricks that run in a specific schedule, and I want it to run my entire notebook, and in the end make it export the final result and save it in a specific directory so I can go back in time and see some old result.
python csv dataframe pyspark databricks
edited Nov 9 at 9:56
Mayank Porwal
3,7181621
asked Nov 9 at 9:50
celllaa95
187
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
Databricks dataFrame
I have a dataFrame that I would like to export the result to a specific path/directory.
By doing display(df), you got a button that you can press that will give you a download off the result that you can open in excel.
How can I do this by using python? I am trying to make a job in databricks that run in a specific schedule, and I want it to run my entire notebook, and in the end make it export the final result and save it in a specific directory so I can go back in time and see some old result.
python csv dataframe pyspark databricks
edited Nov 9 at 9:56
Mayank Porwal
3,7181621
asked Nov 9 at 9:50
celllaa95
187
Databricks dataFrame
I have a dataFrame that I would like to export the result to a specific path/directory.
By doing display(df), you got a button that you can press that will give you a download off the result that you can open in excel.
How can I do this by using python? I am trying to make a job in databricks that run in a specific schedule, and I want it to run my entire notebook, and in the end make it export the final result and save it in a specific directory so I can go back in time and see some old result.
python csv dataframe pyspark databricks
python csv dataframe pyspark databricks
edited Nov 9 at 9:56
Mayank Porwal
3,7181621
asked Nov 9 at 9:50
celllaa95
187
edited Nov 9 at 9:56
Mayank Porwal
3,7181621
asked Nov 9 at 9:50
celllaa95
187
edited Nov 9 at 9:56
Mayank Porwal
3,7181621
edited Nov 9 at 9:56
Mayank Porwal
3,7181621
edited Nov 9 at 9:56
Mayank Porwal
3,7181621
3,7181621
asked Nov 9 at 9:50
celllaa95
187
asked Nov 9 at 9:50
celllaa95
187
asked Nov 9 at 9:50
celllaa95
187
187
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
up vote
0
down vote
You can export the data to a CSV format(Which can be opened in Excel). In order to do so you can convert the DataFrame to a Pandas DataFrame and then convert it to CSV format.
df.toPandas().to_csv("<FILE_DESTINATION>/<FILE_NAME>")
answered Nov 9 at 9:58
Chaitu Pendyala
764
add a comment |
up vote
0
down vote
From what I understand, you want to know how to export a pandas dataframe(csv/ excel) to a directory:
You can use:
import pandas
specific_dir = '/path_of_dir'
df.to_csv(specific_dir+'/mycsv.csv')
df.to_excel(specific_dir+'/myexcel.xlsx')
You can just pass the directory you want to export the dataframe into in the commands above.
answered Nov 9 at 9:59
Mayank Porwal
3,7181621
For some reason, that is not working. I am trying to access a directory that is made in databricks, and I know that I am using the right path, but still it complaints about that the pat does not exist.
– celllaa95
Nov 9 at 10:31
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
You can export the data to a CSV format(Which can be opened in Excel). In order to do so you can convert the DataFrame to a Pandas DataFrame and then convert it to CSV format.
df.toPandas().to_csv("<FILE_DESTINATION>/<FILE_NAME>")
answered Nov 9 at 9:58
Chaitu Pendyala
764
add a comment |
up vote
0
down vote
You can export the data to a CSV format(Which can be opened in Excel). In order to do so you can convert the DataFrame to a Pandas DataFrame and then convert it to CSV format.
df.toPandas().to_csv("<FILE_DESTINATION>/<FILE_NAME>")
answered Nov 9 at 9:58
Chaitu Pendyala
764
add a comment |
up vote
0
down vote
up vote
0
down vote
You can export the data to a CSV format(Which can be opened in Excel). In order to do so you can convert the DataFrame to a Pandas DataFrame and then convert it to CSV format.
df.toPandas().to_csv("<FILE_DESTINATION>/<FILE_NAME>")
answered Nov 9 at 9:58
Chaitu Pendyala
764
You can export the data to a CSV format(Which can be opened in Excel). In order to do so you can convert the DataFrame to a Pandas DataFrame and then convert it to CSV format.
df.toPandas().to_csv("<FILE_DESTINATION>/<FILE_NAME>")
answered Nov 9 at 9:58
Chaitu Pendyala
764
answered Nov 9 at 9:58
Chaitu Pendyala
764
answered Nov 9 at 9:58
Chaitu Pendyala
764
answered Nov 9 at 9:58
Chaitu Pendyala
764
764
add a comment |
add a comment |
up vote
0
down vote
From what I understand, you want to know how to export a pandas dataframe(csv/ excel) to a directory:
You can use:
import pandas
specific_dir = '/path_of_dir'
df.to_csv(specific_dir+'/mycsv.csv')
df.to_excel(specific_dir+'/myexcel.xlsx')
You can just pass the directory you want to export the dataframe into in the commands above.
answered Nov 9 at 9:59
Mayank Porwal
3,7181621
For some reason, that is not working. I am trying to access a directory that is made in databricks, and I know that I am using the right path, but still it complaints about that the pat does not exist.
– celllaa95
Nov 9 at 10:31
add a comment |
up vote
0
down vote
From what I understand, you want to know how to export a pandas dataframe(csv/ excel) to a directory:
You can use:
import pandas
specific_dir = '/path_of_dir'
df.to_csv(specific_dir+'/mycsv.csv')
df.to_excel(specific_dir+'/myexcel.xlsx')
You can just pass the directory you want to export the dataframe into in the commands above.
answered Nov 9 at 9:59
Mayank Porwal
3,7181621
For some reason, that is not working. I am trying to access a directory that is made in databricks, and I know that I am using the right path, but still it complaints about that the pat does not exist.
– celllaa95
Nov 9 at 10:31
add a comment |
up vote
0
down vote
up vote
0
down vote
From what I understand, you want to know how to export a pandas dataframe(csv/ excel) to a directory:
You can use:
import pandas
specific_dir = '/path_of_dir'
df.to_csv(specific_dir+'/mycsv.csv')
df.to_excel(specific_dir+'/myexcel.xlsx')
You can just pass the directory you want to export the dataframe into in the commands above.
answered Nov 9 at 9:59
Mayank Porwal
3,7181621
From what I understand, you want to know how to export a pandas dataframe(csv/ excel) to a directory:
You can use:
import pandas
specific_dir = '/path_of_dir'
df.to_csv(specific_dir+'/mycsv.csv')
df.to_excel(specific_dir+'/myexcel.xlsx')
You can just pass the directory you want to export the dataframe into in the commands above.
answered Nov 9 at 9:59
Mayank Porwal
3,7181621
answered Nov 9 at 9:59
Mayank Porwal
3,7181621
answered Nov 9 at 9:59
Mayank Porwal
3,7181621
answered Nov 9 at 9:59
Mayank Porwal
3,7181621
3,7181621
For some reason, that is not working. I am trying to access a directory that is made in databricks, and I know that I am using the right path, but still it complaints about that the pat does not exist.
– celllaa95
Nov 9 at 10:31
add a comment |
For some reason, that is not working. I am trying to access a directory that is made in databricks, and I know that I am using the right path, but still it complaints about that the pat does not exist.
– celllaa95
Nov 9 at 10:31
For some reason, that is not working. I am trying to access a directory that is made in databricks, and I know that I am using the right path, but still it complaints about that the pat does not exist.
– celllaa95
Nov 9 at 10:31
For some reason, that is not working. I am trying to access a directory that is made in databricks, and I know that I am using the right path, but still it complaints about that the pat does not exist.
– celllaa95
Nov 9 at 10:31
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53223392%2fhow-to-export-dataframe-csv-excel-file-to-a-specific-directory%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
