How to use custom python with existing packages - nix derivation?

Question

I defined a custom nix cpython derivation with unmerged patches. If I specify it as a target nix-shell gives me the version I expect.

{ pkgs ? import <nixpkgs> {} }:
(pkgs.python3.overrideAttrs (old:  {
    src = pkgs.fetchFromGitHub {
      owner = "bergkvist";
      repo = "cpython";
      rev = "01bcf2bef5f4ffffb454da35cb66b186a7a12598";
      sha256 = "1713sx5izd745bgr6fdx6d1g7ivaqy6jrf9v5jgml31bd1nmfccy";
    };
    verions = "3.11.5";
 })) })).override {
    sourceVersion = {major = "3"; minor = "11" ; patch = "5"; suffix = ""; };
  };

$ nix-shell
$ python
Python 3.11.0a0 (default, Aug  4 2021, 00:12:31) [Clang 7.1.0 (tags/RELEASE_710/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

But I don't see a way how to use new cpython for existing packages. Straight forward overlay doesn't affect anything.

{ pkgs ? import <nixpkgs> {} }:
let myPatchedPython = ... ;
    myPkgs = pkgs.extend(self: super: {
      python3 = myPatchedPython;
      python39 = myPatchedPython;
    });
in myPkgs.python39Packages.send2trash

$ nix-shell
$ python3 
Python 3.9.5 (default, Jul 18 2021, 14:31:15)

I noticed I can use override but it is only for runtime and it is not working for all packages.

let  useMyPy = drv: drv.override {
    python = myPatchedPython;
    python3 = myPatchedPython;
    python38 = myPatchedPython;
    python39 = myPatchedPython;
  };
pyobjc-core = useMyPy (pkgs.python39Packages.buildPythonPackage rec {
    pname = "pyobjc-core";
    version = "7.3";
    adfasdf = true;
    name = "${pname}-${version}";
    src = pkgs.python39Packages.fetchPypi {
      pname = "pyobjc-core";
      inherit version;
      sha256 = "0x3msrzvcszlmladdpl64s48l52fwk4xlnnri8daq2mliggsx0ah";
    };
 propagatedBuildInputs = [
      myPatchedPython
      pkgs.darwin.libobjc
      pkgs.darwin.cctools
      pkgs.darwin.apple_sdk.frameworks.Foundation
      pkgs.darwin.apple_sdk.frameworks.AppKit
      pkgs.darwin.apple_sdk.frameworks.CoreServices
      pkgs.darwin.apple_sdk.frameworks.Cocoa
      pkgs.python39Packages.setuptools
    ];
    buildInputs = [ pkgs.libffi ];
  });

in pyobjc-core

$ nix-shell
$ python3
Python 3.11.0a0 (default, Aug  4 2021, 00:12:31) [Clang 7.1.0 (tags/RELEASE_710/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import import AppKit
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nix/store/jcc0xf3zwxnzzh497nip7v57k68r4zb9-python3.9-pyobjc-7.3/lib/python3.9/site-packages/AppKit/__init__.py", line 10, in <module>
    import Foundation
    ^^^^^^^^^^^^^^^^^
...

score 0 · Answer 1 · answered Aug 06 '21 at 17:24

AFAIK, the official way to do this is to define myPatchedPython in an overlay for nixpkgs (in ~/.config/nixpkgs/overlays.nix), and invoke nix-shell like so:

nix-shell -p myPatchedPython

If this does not work, you can also try nix-shell -E '(import <nixpkgs> {}).myPatchedPython' and the like, which give you more of a debugging angle.

The more pedantic way (without overlays) that I have used in the past with success is to directly copy or include your patched Python version in the shell.nix file, e.g.:

{ pkgs ? import <nixpkgs> {} }:
pkgs.mkShell {
  buildInputs = [
    pkgs.python.overrideAttrs { ... }
  ];
}

How to use custom python with existing packages - nix derivation?

1 Answers1

Linked