Neat Python Feature - Pickle!

in python •  7 years ago 

persist.png

Back when I was learning C and C++, I often reused a piece of code that would write data from memory to a file and read it back again to persist state.

Just dump my struct to a file.

Any time the software needed to restart - say after a system crash - it could just keep on trugging along as if not much had happened :)

It is a very long time since I coded in C++ (outside of Arduino and C#) but that concept stuck with me.

I had no idea there was a similar functionality in Python until today (yes, I am late to the party!)

Introducing Pickle

Now if this is not news to you I do not blame you for rolling your eyes, but if you haven't seen Pickle then you are in for a treat.

Start with, as you would expect, importing pickle.

Then use the following code to persist and recall your data:

def persist(item_list):

    with open('my_list.txt', 'wb') as fp:
        pickle.dump(item_list, fp)

Supply this function with your array/list.

And then to get it back ...

def retrieve():

    item_list = []

    try:

        with open('my_list.txt', 'rb') as fp:
            item_list = pickle.load(fp)
    except:
        print("Pickle bork - ah well!")

    return item_list

So what happens is your array or list goes from memory to a file on your file system, and then you can reverse the process :)

Obviously the file needs to exist, so just use the shell command as follows:

touch my_list.txt

(this will create a placeholder file ready for your script to run)

Neat, eh?

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

Pickle is great. I find it most useful when I am dealing with a lot of manually-entered data and an unstable script I wrote off-the-cuff. If I'm an idiot and break something, as long as I keep pickling it to disk as I go, I can just sort of pick it up where I left off.

It’s a really good feature when you are in a pickle and need to get your data back.

Thanks I will be here all week.

2F0922A8-5AE7-4001-BC86-F3238C5D8720.gif

Beginner question:

with open('my_list.txt', 'wb') as fp:

Shouldn't this command create a file if it is not existing already?

Yes but in most cases your scripts will attempt to read first so touching the file is just a precaution, obviously you can check for file existence in your code too and even prepare it with an empty array :)

A similar thing on top of pickle that I use a lot is shelve. It's as easy to use and brings some additional benefits.

Ooh, nice! Thank you :)

Friend had not noticed that Pickle option, which is efficient. Thanks for the input, Greetings ;))

Pickle is highly useful! I'm not sure if I've used it to restore a whole state though. I only really remember using it with big data.

I am using it to check for IDs of records to see if I have seen them before, doesn't sound like a big deal but it saved me some migraines ;)

Thank you for introducing this very useful tool. I may use this to save some unstructured data or model.

What this configuration, from your coding?