##// END OF EJS Templates
packaging: use PyOxidizer for producing WiX MSI installer...
packaging: use PyOxidizer for producing WiX MSI installer We recently taught our in-tree PyOxidizer configuration file to produce MSI installers with WiX using PyOxidizer's built-in support for doing so. This commit changes our WiX + PyOxidizer installer generation code to use this functionality. After this change, all the Python packaging code is doing is the following: * Building HTML documentation * Making gettext available to the build process. * Munging CLI arguments to variables for the `pyoxidizer` execution. * Invoking `pyoxidizer build`. * Copying the produced `.msi` to the `dist/` directory. Applying this stack on stable and rebuilding the 5.8 MSI installer produced the following differences from the official 5.8 installer: * .exe and .pyd files aren't byte identical (this is expected). * Various .dist-info/ directories have different names due to older versions of PyOxidizer being buggy and not properly normalizing package names. (The new behavior is correct.) * Various *.dist-info/RECORD files are different due to content divergence of files (this is expected). * The python38.dll differs due to newer PyOxidizer shipping a newer version of Python 3.8. * We now ship python3.dll because PyOxidizer now includes this file by default. * The vcruntime140.dll differs because newer PyOxidizer installs a newer version. We also now ship a vcruntime140_1.dll because newer versions of the redistributable ship 2 files now. The WiX GUIDs and IDs of installed files have likely changed as a result of PyOxidizer's different mechanism for generating those identifiers. This means that an upgrade install of the MSI will replace files instead of doing an incremental update. This is likely harmless and we've incurred this kind of breakage before. As far as I can tell, the new PyOxidizer-built MSI is functionally equivalent to the old method. Once we drop support for Python 2.7 MSI installers, we can delete the WiX code from the repository. This commit temporarily drops support for extra `.wxs` files. We raise an exception instead of silently not using them, which I think is appropriate. We should be able to add support back in by injecting state into pyoxidizer.bzl via `--var`. I just didn't want to expend cognitive load to think about the solution as part of this series. Differential Revision: https://phab.mercurial-scm.org/D10688

File last commit:

r47574:1977495d default
r47981:73f1a103 default
Show More
Makefile
139 lines | 4.8 KiB | text/x-makefile | MakefileLexer
Augie Fackler
fuzz: if the caller of our makefile sets CC and CXX, trust them...
r47353 CC ?= clang
CXX ?= clang++
Yuya Nishihara
fuzz: expand variables by make...
r38232
Augie Fackler
fuzz: use a more standard approach to allow local builds of fuzzers...
r44265 # By default, use our own standalone_fuzz_target_runner.
# This runner does no fuzzing, but simply executes the inputs
# provided via parameters.
# Run e.g. "make all LIB_FUZZING_ENGINE=/path/to/libFuzzer.a"
# to link the fuzzer(s) against a real fuzzing engine.
#
# OSS-Fuzz will define its own value for LIB_FUZZING_ENGINE.
LIB_FUZZING_ENGINE ?= standalone_fuzz_target_runner.o
Augie Fackler
fuzz: follow modern guidelines and use LIB_FUZZING_ENGINE...
r44262
Martin von Zweigbergk
fuzz: use Python 3 in makefile...
r47574 # Default to Python 3.
#
# Windows ships Python 3 as `python.exe`, which may not be on PATH. py.exe is.
ifeq ($(OS),Windows_NT)
PYTHON?=py -3
else
PYTHON?=python3
endif
Augie Fackler
fuzz: use a variable to allow specifying python-config to use...
r44264 PYTHON_CONFIG ?= $$OUT/sanpy/bin/python-config
Augie Fackler
fuzz: fix Makefile default PYTHON_CONFIG_FLAGS to be modern...
r46885 PYTHON_CONFIG_FLAGS ?= --ldflags --embed
Augie Fackler
fuzz: use a variable to allow specifying python-config to use...
r44264
Augie Fackler
fuzz: suppress deprecated-register warnings in our compile...
r44263 CXXFLAGS += -Wno-deprecated-register
Augie Fackler
tests: finally fix up test-fuzz-targets.t...
r44267 all: standalone_fuzz_target_runner.o oss-fuzz
Yuya Nishihara
fuzz: fix the default make target
r38233
Augie Fackler
fuzz: use a more standard approach to allow local builds of fuzzers...
r44265 standalone_fuzz_target_runner.o: standalone_fuzz_target_runner.cc
Augie Fackler
fuzz: clean up production of seed corpora...
r44295 $$OUT/%_fuzzer_seed_corpus.zip: %_corpus.py
Martin von Zweigbergk
fuzz: use Python 3 in makefile...
r47574 $(PYTHON) $< $@
Augie Fackler
fuzz: clean up production of seed corpora...
r44295
Augie Fackler
fuzz: extract Python initialization to utility package...
r41049 pyutil.o: pyutil.cc pyutil.h
$(CXX) $(CXXFLAGS) -g -O1 \
Augie Fackler
fuzz: use a variable to allow specifying python-config to use...
r44264 `$(PYTHON_CONFIG) --cflags` \
Augie Fackler
fuzz: extract Python initialization to utility package...
r41049 -I../../mercurial -c -o pyutil.o pyutil.cc
Augie Fackler
contrib: add some basic scaffolding for some fuzz test targets...
r35688 bdiff-oss-fuzz.o: ../../mercurial/bdiff.c
Yuya Nishihara
fuzz: expand variables by make...
r38232 $(CC) $(CFLAGS) -c -o bdiff-oss-fuzz.o ../../mercurial/bdiff.c
Augie Fackler
contrib: add some basic scaffolding for some fuzz test targets...
r35688
Augie Fackler
fuzz: clean out most of fuzzutil...
r44013 bdiff_fuzzer: bdiff.cc bdiff-oss-fuzz.o
Yuya Nishihara
fuzz: expand variables by make...
r38232 $(CXX) $(CXXFLAGS) -std=c++17 -I../../mercurial bdiff.cc \
Augie Fackler
fuzz: follow modern guidelines and use LIB_FUZZING_ENGINE...
r44262 bdiff-oss-fuzz.o $(LIB_FUZZING_ENGINE) -o \
Augie Fackler
fuzzutil: make it possible to use absl when C++17 isn't supported...
r38192 $$OUT/bdiff_fuzzer
Augie Fackler
contrib: add some basic scaffolding for some fuzz test targets...
r35688
Augie Fackler
fuzz: new fuzzer for the mpatch code...
r38264 mpatch.o: ../../mercurial/mpatch.c
$(CC) -g -O1 -fsanitize=fuzzer-no-link,address -c -o mpatch.o \
../../mercurial/mpatch.c
mpatch-oss-fuzz.o: ../../mercurial/mpatch.c
$(CC) $(CFLAGS) -c -o mpatch-oss-fuzz.o ../../mercurial/mpatch.c
Augie Fackler
fuzz: clean up production of seed corpora...
r44295 mpatch_fuzzer: mpatch.cc mpatch-oss-fuzz.o $$OUT/mpatch_fuzzer_seed_corpus.zip
Augie Fackler
fuzz: new fuzzer for the mpatch code...
r38264 $(CXX) $(CXXFLAGS) -std=c++17 -I../../mercurial mpatch.cc \
Augie Fackler
fuzz: follow modern guidelines and use LIB_FUZZING_ENGINE...
r44262 mpatch-oss-fuzz.o $(LIB_FUZZING_ENGINE) -o \
Augie Fackler
fuzz: new fuzzer for the mpatch code...
r38264 $$OUT/mpatch_fuzzer
Augie Fackler
fuzz: add a fuzzer for xdiff...
r36697 fuzz-x%.o: ../../mercurial/thirdparty/xdiff/x%.c ../../mercurial/thirdparty/xdiff/*.h
Yuya Nishihara
fuzz: expand variables by make...
r38232 $(CC) $(CFLAGS) -c \
Augie Fackler
fuzz: add a fuzzer for xdiff...
r36697 -o $@ \
$<
Augie Fackler
fuzz: clean out most of fuzzutil...
r44013 xdiff_fuzzer: xdiff.cc fuzz-xdiffi.o fuzz-xprepare.o fuzz-xutils.o
Yuya Nishihara
fuzz: expand variables by make...
r38232 $(CXX) $(CXXFLAGS) -std=c++17 -I../../mercurial xdiff.cc \
Augie Fackler
fuzz: clean out most of fuzzutil...
r44013 fuzz-xdiffi.o fuzz-xprepare.o fuzz-xutils.o \
Augie Fackler
fuzz: follow modern guidelines and use LIB_FUZZING_ENGINE...
r44262 $(LIB_FUZZING_ENGINE) -o $$OUT/xdiff_fuzzer
Augie Fackler
fuzz: add a fuzzer for xdiff...
r36697
Augie Fackler
fuzz: clean up some repetition on building parsers.so fuzzers...
r44269 parsers-%.o: ../../mercurial/cext/%.c
$(CC) -I../../mercurial `$(PYTHON_CONFIG) --cflags` $(CFLAGS) -c \
-o $@ $<
Augie Fackler
fuzz: new fuzzer for cext/manifest.c...
r40089
Augie Fackler
fuzz: clean up some repetition on building parsers.so fuzzers...
r44269 PARSERS_OBJS=parsers-manifest.o parsers-charencode.o parsers-parsers.o parsers-dirs.o parsers-pathencode.o parsers-revlog.o
Augie Fackler
fuzz: new fuzzer for cext/manifest.c...
r40089
Augie Fackler
fuzz: add a seed corpus for the dirs fuzzer...
r44296 dirs_fuzzer: dirs.cc pyutil.o $(PARSERS_OBJS) $$OUT/dirs_fuzzer_seed_corpus.zip
Augie Fackler
fuzz: use a variable to allow specifying python-config to use...
r44264 $(CXX) $(CXXFLAGS) `$(PYTHON_CONFIG) --cflags` \
Augie Fackler
fuzz: new fuzzer for dirs.c...
r43420 -Wno-register -Wno-macro-redefined \
-I../../mercurial dirs.cc \
Augie Fackler
fuzz: clean up some repetition on building parsers.so fuzzers...
r44269 pyutil.o $(PARSERS_OBJS) \
Augie Fackler
fuzz: add config knob for PYTHON_CONFIG_FLAGS...
r45489 $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
Augie Fackler
fuzz: new fuzzer for dirs.c...
r43420 -o $$OUT/dirs_fuzzer
Augie Fackler
fuzz: clean up some repetition on building parsers.so fuzzers...
r44269 fncache_fuzzer: fncache.cc
Augie Fackler
fuzz: use a variable to allow specifying python-config to use...
r44264 $(CXX) $(CXXFLAGS) `$(PYTHON_CONFIG) --cflags` \
Augie Fackler
fuzz: new fuzzer for fncache-related functions...
r43422 -Wno-register -Wno-macro-redefined \
-I../../mercurial fncache.cc \
Augie Fackler
fuzz: clean up some repetition on building parsers.so fuzzers...
r44269 pyutil.o $(PARSERS_OBJS) \
Augie Fackler
fuzz: add config knob for PYTHON_CONFIG_FLAGS...
r45489 $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
Augie Fackler
fuzz: new fuzzer for fncache-related functions...
r43422 -o $$OUT/fncache_fuzzer
Augie Fackler
fuzz: clean up some repetition on building parsers.so fuzzers...
r44269 jsonescapeu8fast_fuzzer: jsonescapeu8fast.cc pyutil.o $(PARSERS_OBJS)
Augie Fackler
fuzz: use a variable to allow specifying python-config to use...
r44264 $(CXX) $(CXXFLAGS) `$(PYTHON_CONFIG) --cflags` \
Augie Fackler
fuzz: new target to fuzz jsonescapeu8fast...
r43423 -Wno-register -Wno-macro-redefined \
-I../../mercurial jsonescapeu8fast.cc \
Augie Fackler
fuzz: clean up some repetition on building parsers.so fuzzers...
r44269 pyutil.o $(PARSERS_OBJS) \
Augie Fackler
fuzz: add config knob for PYTHON_CONFIG_FLAGS...
r45489 $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
Augie Fackler
fuzz: new target to fuzz jsonescapeu8fast...
r43423 -o $$OUT/jsonescapeu8fast_fuzzer
Augie Fackler
fuzz: clean up production of seed corpora...
r44295 manifest_fuzzer: manifest.cc pyutil.o $(PARSERS_OBJS) $$OUT/manifest_fuzzer_seed_corpus.zip
Augie Fackler
fuzz: use a variable to allow specifying python-config to use...
r44264 $(CXX) $(CXXFLAGS) `$(PYTHON_CONFIG) --cflags` \
Augie Fackler
fuzz: new fuzzer for cext/manifest.c...
r40089 -Wno-register -Wno-macro-redefined \
-I../../mercurial manifest.cc \
Augie Fackler
fuzz: clean up some repetition on building parsers.so fuzzers...
r44269 pyutil.o $(PARSERS_OBJS) \
Augie Fackler
fuzz: add config knob for PYTHON_CONFIG_FLAGS...
r45489 $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
Augie Fackler
fuzz: new fuzzer for cext/manifest.c...
r40089 -o $$OUT/manifest_fuzzer
Augie Fackler
fuzz: clean up production of seed corpora...
r44295 revlog_fuzzer: revlog.cc pyutil.o $(PARSERS_OBJS) $$OUT/revlog_fuzzer_seed_corpus.zip
Augie Fackler
fuzz: use a variable to allow specifying python-config to use...
r44264 $(CXX) $(CXXFLAGS) `$(PYTHON_CONFIG) --cflags` \
Augie Fackler
fuzz: new fuzzer for revlog's parse_index2 method...
r41050 -Wno-register -Wno-macro-redefined \
-I../../mercurial revlog.cc \
Augie Fackler
fuzz: clean up some repetition on building parsers.so fuzzers...
r44269 pyutil.o $(PARSERS_OBJS) \
Augie Fackler
fuzz: add config knob for PYTHON_CONFIG_FLAGS...
r45489 $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
Augie Fackler
fuzz: new fuzzer for revlog's parse_index2 method...
r41050 -o $$OUT/revlog_fuzzer
Augie Fackler
fuzz: clean up production of seed corpora...
r44295 dirstate_fuzzer: dirstate.cc pyutil.o $(PARSERS_OBJS) $$OUT/dirstate_fuzzer_seed_corpus.zip
Augie Fackler
fuzz: use a variable to allow specifying python-config to use...
r44264 $(CXX) $(CXXFLAGS) `$(PYTHON_CONFIG) --cflags` \
Augie Fackler
fuzz: new fuzzer for dirstate parser...
r41051 -Wno-register -Wno-macro-redefined \
-I../../mercurial dirstate.cc \
Augie Fackler
fuzz: clean up some repetition on building parsers.so fuzzers...
r44269 pyutil.o $(PARSERS_OBJS) \
Augie Fackler
fuzz: add config knob for PYTHON_CONFIG_FLAGS...
r45489 $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
Augie Fackler
fuzz: new fuzzer for dirstate parser...
r41051 -o $$OUT/dirstate_fuzzer
Augie Fackler
fuzz: clean up production of seed corpora...
r44295 fm1readmarkers_fuzzer: fm1readmarkers.cc pyutil.o $(PARSERS_OBJS) $$OUT/fm1readmarkers_fuzzer_seed_corpus.zip
Augie Fackler
fuzz: use a variable to allow specifying python-config to use...
r44264 $(CXX) $(CXXFLAGS) `$(PYTHON_CONFIG) --cflags` \
Augie Fackler
fuzz: new fuzzer for parsers.fm1readmarkers...
r41053 -Wno-register -Wno-macro-redefined \
-I../../mercurial fm1readmarkers.cc \
Augie Fackler
fuzz: clean up some repetition on building parsers.so fuzzers...
r44269 pyutil.o $(PARSERS_OBJS) \
Augie Fackler
fuzz: add config knob for PYTHON_CONFIG_FLAGS...
r45489 $(LIB_FUZZING_ENGINE) `$(PYTHON_CONFIG) $(PYTHON_CONFIG_FLAGS)` \
Augie Fackler
fuzz: new fuzzer for parsers.fm1readmarkers...
r41053 -o $$OUT/fm1readmarkers_fuzzer
Augie Fackler
fuzz: add clean target...
r38193 clean:
Yuya Nishihara
fuzz: fix "make clean" to pass even if no binaries built yet
r38236 $(RM) *.o *_fuzzer \
Augie Fackler
fuzz: add clean target...
r38193 bdiff \
Augie Fackler
fuzz: new fuzzer for the mpatch code...
r38264 mpatch \
Augie Fackler
fuzz: add clean target...
r38193 xdiff
Augie Fackler
fuzz: clean up production of seed corpora...
r44295 oss-fuzz: bdiff_fuzzer mpatch_fuzzer xdiff_fuzzer dirs_fuzzer fncache_fuzzer jsonescapeu8fast_fuzzer manifest_fuzzer revlog_fuzzer dirstate_fuzzer fm1readmarkers_fuzzer
Augie Fackler
contrib: add some basic scaffolding for some fuzz test targets...
r35688
Augie Fackler
fuzz: stop building Python in the Makefile...
r41219 .PHONY: all clean oss-fuzz