Create script to generate Flatpak manifest from PIP packages

The tool to generate flatpaks, flatpak-builder is designed to be sandboxed and generate reproducible results; The downside to this is that some existing tooling such as npm or pip don't necessarily map well to this as they by default download dependencies and packages generally have a lot of dependencies which you wouldn't want to list by hand.

So the solution to this is simply to have automated tooling to generate a json file containing all of the dependencies and include them in your manifest.

The information we need to do this is the list of packages, their sha256, and a URL to download them. Thankfully pip has a download option which resolves the list of all dependencies and downloads them for us so we can get the hash ourselves.

To get the download URL we have to use pypi's API at https://pypi.python.org/pypi/{package_name}/json and find the releases url that matches the filename pip grabbed earlier which is easy enough.

Now that flatpak-builder can reliably download static versions of dependencies we need to install it. The pip install tool does allow looking up packages in a local directory so we can pass it --no-index --find-links "file://${PWD}".

This solution did not turn out perfectly though. For a reason I've yet to track down some packages, for example mypy, have actual dependencies that are not listed or pulled in which is frustrating and currently just worked around by allowing you to pass multiple packages at once so you can manually specify anything missing which seems acceptable for now.

The end result looks like this:

{
    "name": "jedi",
    "buildsystem": "simple",
    "build-commands": [
        "pip3 install --no-index --find-links \"file://${PWD}\" --prefix=/app jedi"
    ],
    "sources": [
        {
            "type": "file",
            "url": "https://pypi.python.org/packages/14/3b/15cfd9c7a8bd9e3a2fe956e20fbc4e7c5768e06aea347d8eb68a05a71653/parso-0.1.1.tar.gz",
            "sha256": "5815f3fe254e5665f3c5d6f54f086c2502035cb631a91341591b5a564203cffb"
        },
        {
            "type": "file",
            "url": "https://pypi.python.org/packages/d2/41/430b325e411d564b1afc37bc7286c3549f4c415ada750a617fb1943c593d/jedi-0.11.1.tar.gz",
            "sha256": "d6e799d04d1ade9459ed0f20de47c32f2285438956a677d083d3c98def59fa97"
        }
    ]
}

Merge request: https://github.com/flatpak/flatpak-builder-tools/pull/6

---

Posted on Utopian.io - Rewarding Open Source Contributors

---