In this installment of Bronto Support’s most frequently asked questions, we’ll take a closer look at Data Loader and the best practices we recommend to make uploading your data run as smoothly as possible. While we would love to think we live in an idyllic world of infallible networks, the truth is networks do fail, and not everything always goes as planned. That’s where we tend to come in. When problems arise, we’re just a chat or phone call away. But when it comes to these types of issues, it can often be tough to determine just where things went wrong and could even require some extra work for the user to get things up and running again. So to be better prepared for the inevitable, we recommend these two steps when uploading your precious data:
- Upload a file with a unique filename every time.
- Set up logging for your FTP transaction.
Upload a File With a Unique Filename Every Time
Bronto allows you to upload a file with a static name (e.g. my_orders.csv), but did you know you can specify a pattern and send a unique file each day or hour (e.g. my_orders_062916.csv)? By using unique names, if you ever have an issue uploading a file, you can upload both files the next day and have them process normally. But you can't do this if you are using a static filename. You would be forced to merge the data into one file, requiring more work for you. A Filename Matching Scheme of my_orders*.csv
can process my_orders_062816.csv AND my_orders_062916.csv on the same day. Read more about this feature at Filename Matching Schemes in Bronto Help & Support.
Set Up Logging for Your FTP Transaction
This step is essential for troubleshooting what went wrong. Logging at the FTP level allows you to verify if the file was successfully uploaded and helps resolve any network-related issues. There are a large number of FTP clients on the market, and we cannot describe how to set them all up, but here are a couple that may point you in the right direction: *NIX SCP/SFTP
If you have a script using scp or sftp, you redirect output the same way you would normally redirect output with most command line tools. Here is an example of SCP using the -v option for a more verbose log:Windows: Putty's PSCP/PSFTP
Putty has options you can use to log the output of your FTP session. Version 0.66 and greater of PSCP and PLINK include the -sshlog file option, which you can use to create a log file.
Are You Ready?
Following these tips will give you the protection you need against a network-related failure and make it easier to get back on track if it happens. But remember: You can always contact us Monday-Friday 3 a.m. – 11 p.m.
with any questions or concerns. We are happy to help! That’s it for now, but stay tuned for future posts with more FAQs and best practices from your friendly neighborhood support team here at Bronto. Until then, happy data loading!