<86>Aug 22 09:00:55 userdel[3127286]: delete user 'rooter' <86>Aug 22 09:00:55 groupadd[3127297]: group added to /etc/group: name=rooter, GID=537 <86>Aug 22 09:00:55 groupadd[3127297]: group added to /etc/gshadow: name=rooter <86>Aug 22 09:00:55 groupadd[3127297]: new group: name=rooter, GID=537 <86>Aug 22 09:00:55 useradd[3127308]: new user: name=rooter, UID=537, GID=537, home=/root, shell=/bin/bash <86>Aug 22 09:00:55 userdel[3127317]: delete user 'builder' <86>Aug 22 09:00:55 groupadd[3127322]: group added to /etc/group: name=builder, GID=538 <86>Aug 22 09:00:55 groupadd[3127322]: group added to /etc/gshadow: name=builder <86>Aug 22 09:00:55 groupadd[3127322]: new group: name=builder, GID=538 <86>Aug 22 09:00:55 useradd[3127326]: new user: name=builder, UID=538, GID=538, home=/usr/src, shell=/bin/bash /usr/src/in/srpm/python3-module-blosc-1.5.1-alt3.src.rpm: bad symbols in the license tag: / <13>Aug 22 09:01:02 rpmi: libgdbm-1.8.3-alt10 1454943334 installed <13>Aug 22 09:01:02 rpmi: libexpat-2.2.9-alt1 sisyphus+252464.200.2.1 1590958865 installed <13>Aug 22 09:01:02 rpmi: libp11-kit-0.23.15-alt2 sisyphus+252784.100.2.2 1591274901 installed <13>Aug 22 09:01:02 rpmi: libtasn1-4.16.0-alt1 sisyphus+245480.100.1.1 1580825062 installed <13>Aug 22 09:01:02 rpmi: rpm-macros-alternatives-0.5.1-alt1 sisyphus+226946.100.1.1 1554830426 installed <13>Aug 22 09:01:02 rpmi: alternatives-0.5.1-alt1 sisyphus+226946.100.1.1 1554830426 installed <13>Aug 22 09:01:02 rpmi: ca-certificates-2020.06.29-alt1 sisyphus+254237.300.1.1 1593450881 installed <13>Aug 22 09:01:02 rpmi: ca-trust-0.1.2-alt1 sisyphus+233348.100.1.1 1561653823 installed <13>Aug 22 09:01:02 rpmi: p11-kit-trust-0.23.15-alt2 sisyphus+252784.100.2.2 1591274901 installed <13>Aug 22 09:01:02 rpmi: libcrypto1.1-1.1.1g-alt1 sisyphus+249982.60.8.1 1587743711 installed <13>Aug 22 09:01:02 rpmi: libssl1.1-1.1.1g-alt1 sisyphus+249982.60.8.1 1587743711 installed <13>Aug 22 09:01:02 rpmi: python3-3.8.5-alt1 sisyphus+244405.100.3.1 1595544514 installed <13>Aug 22 09:01:04 rpmi: python3-base-3.8.5-alt1 sisyphus+244405.100.3.1 1595544514 installed <13>Aug 22 09:01:04 rpmi: libpython3-3.8.5-alt1 sisyphus+244405.100.3.1 1595544514 installed <13>Aug 22 09:01:04 rpmi: tests-for-installed-python3-pkgs-0.1.13.1-alt2 1535450458 installed <13>Aug 22 09:01:04 rpmi: rpm-build-python3-0.1.13.1-alt2 1535450458 installed <13>Aug 22 09:01:13 rpmi: python3-module-pkg_resources-1:46.1.3-alt1 sisyphus+250566.200.3.1 1587973342 installed <13>Aug 22 09:01:13 rpmi: python3-module-six-1.15.0-alt1 sisyphus+255738.100.2.1 1596527214 installed <13>Aug 22 09:01:13 rpmi: python3-module-ntlm-1.1.0-alt1.2 sisyphus+228512.100.1.1 1556654575 installed <13>Aug 22 09:01:13 rpmi: python3-module-snowballstemmer-1.2.0-alt2.1 1457859319 installed <13>Aug 22 09:01:13 rpmi: python3-module-pyparsing-2.4.2-alt1 sisyphus+236061.100.3.1 1565853577 installed <13>Aug 22 09:01:13 rpmi: python3-module-packaging-19.0-alt2 sisyphus+235493.3500.8.1 1565461874 installed <13>Aug 22 09:01:13 rpmi: python3-module-markupsafe-1.1.1-alt1 sisyphus+248369.100.1.1 1585046136 installed <13>Aug 22 09:01:13 rpmi: python3-module-jinja2-2.11.2-alt1 sisyphus+254573.100.1.1 1594043344 installed <13>Aug 22 09:01:13 rpmi: python3-module-imagesize-1.1.0-alt1 sisyphus+229015.100.2.1 1557233756 installed <13>Aug 22 09:01:13 rpmi: python3-module-pytz-1:2020.1-alt1 sisyphus+254576.100.1.1 1594046074 installed <13>Aug 22 09:01:14 rpmi: python3-module-babel-1:2.6.0-alt1 sisyphus+228351.2400.6.2 1556652168 installed <13>Aug 22 09:01:14 rpmi: python3-module-ndg-0.4.2-alt1.qa1 sisyphus+227504.1300.1.2 1555853234 installed <13>Aug 22 09:01:14 rpmi: python3-module-idna-2.10-alt1 sisyphus+255040.200.1.1 1594995891 installed <13>Aug 22 09:01:14 rpmi: python3-module-chardet-3.0.4-alt1 sisyphus+227476.1700.1.2 1555756717 installed <13>Aug 22 09:01:14 rpmi: python3-module-webencodings-0.5.1-alt2 sisyphus+245915.200.1.1 1581496105 installed <13>Aug 22 09:01:15 rpmi: python3-module-genshi-0.7-alt2 sisyphus+229363.100.1.1 1557847335 installed <13>Aug 22 09:01:15 rpmi: python3-module-pycparser-2.19-alt2 sisyphus+245734.100.1.1 1581192421 installed <13>Aug 22 09:01:15 rpmi: python3-module-cffi-1.14.0-alt1 sisyphus+255743.100.1.1 1596453449 installed <13>Aug 22 09:01:15 rpmi: python3-module-cryptography-3.0-alt2 sisyphus+255750.200.1.1 1596457913 installed <13>Aug 22 09:01:15 rpmi: python3-module-OpenSSL-19.0.0-alt1 sisyphus+238757.100.1.1 1570339901 installed <13>Aug 22 09:01:15 rpmi: python3-module-ndg-httpsclient-0.4.2-alt1.qa1 sisyphus+227504.1300.1.2 1555853234 installed <13>Aug 22 09:01:15 rpmi: python3-module-urllib3-2:1.25.10-alt1 sisyphus+255342.100.1.1 1595592746 installed <13>Aug 22 09:01:15 rpmi: python3-module-requests-2.24.0-alt1 sisyphus+254580.100.1.1 1594048060 installed <13>Aug 22 09:01:15 rpmi: python-sphinx-objects.inv-1:2.3.10.20200814-alt1 sisyphus+256437.100.1.1 1597793855 installed <13>Aug 22 09:01:15 rpmi: libgpg-error-1.36-alt1 sisyphus+225621.300.1.1 1553521082 installed <13>Aug 22 09:01:15 rpmi: libgcrypt20-1.8.5-alt3 sisyphus+239622.100.1.1 1571746654 installed <13>Aug 22 09:01:15 rpmi: libxslt-1.1.34-alt2 sisyphus+248264.100.1.1 1584829770 installed <13>Aug 22 09:01:15 rpmi: python3-module-cssselect-0.9.1-alt2 sisyphus+250566.2300.6.1 1588188959 installed <13>Aug 22 09:01:15 rpmi: python3-module-html5lib-1:1.0.1-alt1 sisyphus+238807.100.2.1 1570465973 installed <13>Aug 22 09:01:16 rpmi: python3-module-lxml-4.5.0-alt2 sisyphus+250566.2700.6.1 1588189778 installed <13>Aug 22 09:01:16 rpmi: python3-module-docutils-0.16.0-alt1 sisyphus+250087.100.1.1 1587043829 installed <13>Aug 22 09:01:16 rpmi: python3-module-alabaster-0.7.6-alt3 sisyphus+228351.700.4.1 1556637370 installed <13>Aug 22 09:01:16 rpmi: python3-module-sphinxcontrib-applehelp-1.0.2-alt1 sisyphus+252614.200.1.1 1590949678 installed <13>Aug 22 09:01:16 rpmi: python3-module-sphinxcontrib-devhelp-1.0.2-alt1 sisyphus+252614.300.1.1 1590949711 installed <13>Aug 22 09:01:16 rpmi: python3-module-sphinxcontrib-htmlhelp-1.0.2-alt1 sisyphus+228401.100.1.1 1556572721 installed <13>Aug 22 09:01:16 rpmi: python3-module-sphinxcontrib-jsmath-1.0.1-alt1 sisyphus+238085.500.1.1 1569313099 installed <13>Aug 22 09:01:16 rpmi: python3-module-sphinxcontrib-qthelp-1.0.3-alt1 sisyphus+252614.400.1.1 1590949745 installed <13>Aug 22 09:01:16 rpmi: python3-module-sphinxcontrib-serializinghtml-1.1.4-alt1 sisyphus+252614.100.1.1 1590949645 installed <13>Aug 22 09:01:17 rpmi: python3-module-sphinx-1:2.4.4-alt1 sisyphus+254723.100.1.2 1594318607 installed <13>Aug 22 09:01:17 rpmi: python3-module-Pygments-2.6.1-alt1 sisyphus+251222.200.1.1 1588756845 installed <13>Aug 22 09:01:17 rpmi: libsnappy-1.1.7-alt1 sisyphus+227320.100.1.3 1555618694 installed <13>Aug 22 09:01:17 rpmi: liblz4-1:1.9.2-alt1 sisyphus+238585.100.2.2 1570066927 installed <13>Aug 22 09:01:17 rpmi: libblosc-1.18.1-alt1 sisyphus+250012.100.1.1 1586944162 installed <13>Aug 22 09:01:17 rpmi: libblosc-devel-1.18.1-alt1 sisyphus+250012.100.1.1 1586944162 installed <13>Aug 22 09:01:17 rpmi: python3-module-numpydoc-1:0.7.0-alt1 sisyphus+227506.500.1.2 1555852058 installed Building target platforms: i586 Building for target i586 Wrote: /usr/src/in/nosrpm/python3-module-blosc-1.5.1-alt3.nosrc.rpm <13>Aug 22 09:01:24 rpmi: libtinfo-devel-6.1.20180407-alt2 sisyphus+222164.200.1.1 1550686226 installed <13>Aug 22 09:01:24 rpmi: libncurses-devel-6.1.20180407-alt2 sisyphus+222164.200.1.1 1550686226 installed <13>Aug 22 09:01:24 rpmi: python3-dev-3.8.5-alt1 sisyphus+244405.100.3.1 1595544514 installed <13>Aug 22 09:01:24 rpmi: python3-module-setuptools-1:46.1.3-alt1 sisyphus+250566.200.3.1 1587973342 installed Installing python3-module-blosc-1.5.1-alt3.src.rpm Building target platforms: i586 Building for target i586 Executing(%prep): /bin/sh -e /usr/src/tmp/rpm-tmp.9435 + umask 022 + /bin/mkdir -p /usr/src/RPM/BUILD + cd /usr/src/RPM/BUILD + cd /usr/src/RPM/BUILD + rm -rf python3-module-blosc-1.5.1 + echo 'Source #0 (python3-module-blosc-1.5.1.tar):' Source #0 (python3-module-blosc-1.5.1.tar): + /bin/tar -xf /usr/src/RPM/SOURCES/python3-module-blosc-1.5.1.tar + cd python3-module-blosc-1.5.1 + /bin/chmod -c -Rf u+rwX,go-w . + sed -i 's|sphinx-build|sphinx-build-3|' doc/Makefile ++ find ./ -name '*.py' + sed -i 's|#!/usr/bin/env python|#!/usr/bin/env python3|' ./setup.py ./doc/conf.py ./cpuinfo.py ./c-blosc/bench/plot-speeds.py ./blosc/toplevel.py ./blosc/test.py ./blosc/__init__.py ./bench/threadpool.py ./bench/compress_ptr.py ./bench/compare-pack-ptr.py + exit 0 Executing(%build): /bin/sh -e /usr/src/tmp/rpm-tmp.59204 + umask 022 + /bin/mkdir -p /usr/src/RPM/BUILD + cd /usr/src/RPM/BUILD + cd python3-module-blosc-1.5.1 + CFLAGS='-pipe -frecord-gcc-switches -Wall -g -O2 -march=i586 -mtune=generic' + export CFLAGS + CXXFLAGS='-pipe -frecord-gcc-switches -Wall -g -O2 -march=i586 -mtune=generic' + export CXXFLAGS + FFLAGS='-pipe -frecord-gcc-switches -Wall -g -O2 -march=i586 -mtune=generic' + export FFLAGS + /usr/bin/python3 setup.py build --debug --blosc=/usr running build running build_py creating build creating build/lib.linux-i686-3.8 creating build/lib.linux-i686-3.8/blosc copying blosc/version.py -> build/lib.linux-i686-3.8/blosc copying blosc/__init__.py -> build/lib.linux-i686-3.8/blosc copying blosc/test.py -> build/lib.linux-i686-3.8/blosc copying blosc/toplevel.py -> build/lib.linux-i686-3.8/blosc running build_ext building 'blosc.blosc_extension' extension creating build/temp.linux-i686-3.8 creating build/temp.linux-i686-3.8/blosc i586-alt-linux-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -pipe -frecord-gcc-switches -Wall -g -O3 -march=i586 -mtune=generic -pipe -frecord-gcc-switches -Wall -g -O2 -march=i586 -mtune=generic -fPIC -g -I/usr/include -I/usr/include/python3.8 -c blosc/blosc_extension.c -o build/temp.linux-i686-3.8/blosc/blosc_extension.o -pipe -frecord-gcc-switches -Wall -g -O2 -march=i586 -mtune=generic blosc/blosc_extension.c: In function ‘PyBlosc_compressor_list’: blosc/blosc_extension.c:88:8: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers] 88 | list = blosc_list_compressors(); | ^ blosc/blosc_extension.c: In function ‘PyBlosc_code_to_name’: blosc/blosc_extension.c:107:40: warning: passing argument 2 of ‘blosc_compcode_to_compname’ from incompatible pointer type [-Wincompatible-pointer-types] 107 | if (blosc_compcode_to_compname(code, &name) < 0) | ^~~~~ | | | char ** In file included from blosc/blosc_extension.c:13: /usr/include/blosc.h:389:72: note: expected ‘const char **’ but argument is of type ‘char **’ 389 | BLOSC_EXPORT int blosc_compcode_to_compname(int compcode, const char **compname); | ~~~~~~~~~~~~~^~~~~~~~ blosc/blosc_extension.c: In function ‘PyBlosc_get_clib’: blosc/blosc_extension.c:329:8: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers] 329 | clib = blosc_cbuffer_complib(input); | ^ i586-alt-linux-gcc -pthread -shared -pipe -frecord-gcc-switches -Wall -g -O2 -march=i586 -mtune=generic -g build/temp.linux-i686-3.8/blosc/blosc_extension.o -L/usr/lib -L/usr/lib -lblosc -o build/lib.linux-i686-3.8/blosc/blosc_extension.cpython-38.so + exit 0 Executing(%install): /bin/sh -e /usr/src/tmp/rpm-tmp.59421 + umask 022 + /bin/mkdir -p /usr/src/RPM/BUILD + cd /usr/src/RPM/BUILD + /bin/chmod -Rf u+rwX -- /usr/src/tmp/python3-module-blosc-buildroot + : + /bin/rm -rf -- /usr/src/tmp/python3-module-blosc-buildroot + cd python3-module-blosc-1.5.1 + CFLAGS='-pipe -frecord-gcc-switches -Wall -g -O2 -march=i586 -mtune=generic' + export CFLAGS + CXXFLAGS='-pipe -frecord-gcc-switches -Wall -g -O2 -march=i586 -mtune=generic' + export CXXFLAGS + FFLAGS='-pipe -frecord-gcc-switches -Wall -g -O2 -march=i586 -mtune=generic' + export FFLAGS + /usr/bin/python3 setup.py install --skip-build --root=/usr/src/tmp/python3-module-blosc-buildroot --force --blosc=/usr running install running install_lib creating /usr/src/tmp/python3-module-blosc-buildroot creating /usr/src/tmp/python3-module-blosc-buildroot/usr creating /usr/src/tmp/python3-module-blosc-buildroot/usr/lib creating /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3 creating /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages creating /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc copying build/lib.linux-i686-3.8/blosc/blosc_extension.cpython-38.so -> /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc copying build/lib.linux-i686-3.8/blosc/toplevel.py -> /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc copying build/lib.linux-i686-3.8/blosc/test.py -> /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc copying build/lib.linux-i686-3.8/blosc/__init__.py -> /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc copying build/lib.linux-i686-3.8/blosc/version.py -> /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc byte-compiling /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/toplevel.py to toplevel.cpython-38.pyc byte-compiling /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py to test.cpython-38.pyc byte-compiling /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/__init__.py to __init__.cpython-38.pyc byte-compiling /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/version.py to version.cpython-38.pyc running install_egg_info running egg_info creating blosc.egg-info writing blosc.egg-info/PKG-INFO writing dependency_links to blosc.egg-info/dependency_links.txt writing top-level names to blosc.egg-info/top_level.txt writing manifest file 'blosc.egg-info/SOURCES.txt' reading manifest file 'blosc.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching '*.cpp' under directory 'c-blosc' warning: no files found matching '*.hpp' under directory 'c-blosc' writing manifest file 'blosc.egg-info/SOURCES.txt' Copying blosc.egg-info to /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc-1.5.1-py3.8.egg-info running install_scripts + export PYTHONPATH=/usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages + PYTHONPATH=/usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages + make -C doc pickle make: Entering directory '/usr/src/RPM/BUILD/python3-module-blosc-1.5.1/doc' sphinx-build-3 -b pickle -d _build/doctrees -n -v -v . _build/pickle Running Sphinx v2.4.4 [app] setting up extension: 'sphinx.addnodes' [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] setting up extension: 'sphinx.builders.changes' [app] adding builder: [app] setting up extension: 'sphinx.builders.epub3' [app] adding builder: [app] adding config value: ('epub_basename', . at 0xf64f2580>, None) [app] adding config value: ('epub_version', 3.0, 'epub') [app] adding config value: ('epub_theme', 'epub', 'epub') [app] adding config value: ('epub_theme_options', {}, 'epub') [app] adding config value: ('epub_title', . at 0xf5f79d18>, 'epub') [app] adding config value: ('epub_author', . at 0xf5f79d60>, 'epub') [app] adding config value: ('epub_language', . at 0xf5f79da8>, 'epub') [app] adding config value: ('epub_publisher', . at 0xf5f79df0>, 'epub') [app] adding config value: ('epub_copyright', . at 0xf5f79e38>, 'epub') [app] adding config value: ('epub_identifier', 'unknown', 'epub') [app] adding config value: ('epub_scheme', 'unknown', 'epub') [app] adding config value: ('epub_uid', 'unknown', 'env') [app] adding config value: ('epub_cover', (), 'env') [app] adding config value: ('epub_guide', (), 'env') [app] adding config value: ('epub_pre_files', [], 'env') [app] adding config value: ('epub_post_files', [], 'env') [app] adding config value: ('epub_css_files', . at 0xf5f79e80>, 'epub') [app] adding config value: ('epub_exclude_files', [], 'env') [app] adding config value: ('epub_tocdepth', 3, 'env') [app] adding config value: ('epub_tocdup', True, 'env') [app] adding config value: ('epub_tocscope', 'default', 'env') [app] adding config value: ('epub_fix_images', False, 'env') [app] adding config value: ('epub_max_image_width', 0, 'env') [app] adding config value: ('epub_show_urls', 'inline', 'epub') [app] adding config value: ('epub_use_index', . at 0xf5f79ec8>, 'epub') [app] adding config value: ('epub_description', 'unknown', 'epub') [app] adding config value: ('epub_contributor', 'unknown', 'epub') [app] adding config value: ('epub_writing_mode', 'horizontal', 'epub', ) [app] connecting event 'config-inited': [id=0] [app] connecting event 'builder-inited': [id=1] [app] setting up extension: 'sphinx.builders.dirhtml' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding config value: ('html_theme', 'alabaster', 'html') [app] adding config value: ('html_theme_path', [], 'html') [app] adding config value: ('html_theme_options', {}, 'html') [app] adding config value: ('html_title', . at 0xf5f79fa0>, 'html', []) [app] adding config value: ('html_short_title', . at 0xf5f7f028>, 'html') [app] adding config value: ('html_style', None, 'html', []) [app] adding config value: ('html_logo', None, 'html', []) [app] adding config value: ('html_favicon', None, 'html', []) [app] adding config value: ('html_css_files', [], 'html') [app] adding config value: ('html_js_files', [], 'html') [app] adding config value: ('html_static_path', [], 'html') [app] adding config value: ('html_extra_path', [], 'html') [app] adding config value: ('html_last_updated_fmt', None, 'html', []) [app] adding config value: ('html_sidebars', {}, 'html') [app] adding config value: ('html_additional_pages', {}, 'html') [app] adding config value: ('html_domain_indices', True, 'html', []) [app] adding config value: ('html_add_permalinks', '¶', 'html') [app] adding config value: ('html_use_index', True, 'html') [app] adding config value: ('html_split_index', False, 'html') [app] adding config value: ('html_copy_source', True, 'html') [app] adding config value: ('html_show_sourcelink', True, 'html') [app] adding config value: ('html_sourcelink_suffix', '.txt', 'html') [app] adding config value: ('html_use_opensearch', '', 'html') [app] adding config value: ('html_file_suffix', None, 'html', []) [app] adding config value: ('html_link_suffix', None, 'html', []) [app] adding config value: ('html_show_copyright', True, 'html') [app] adding config value: ('html_show_sphinx', True, 'html') [app] adding config value: ('html_context', {}, 'html') [app] adding config value: ('html_output_encoding', 'utf-8', 'html') [app] adding config value: ('html_compact_lists', True, 'html') [app] adding config value: ('html_secnumber_suffix', '. ', 'html') [app] adding config value: ('html_search_language', None, 'html', []) [app] adding config value: ('html_search_options', {}, 'html') [app] adding config value: ('html_search_scorer', '', None) [app] adding config value: ('html_scaled_image_link', True, 'html') [app] adding config value: ('html_baseurl', '', 'html') [app] adding config value: ('html_math_renderer', None, 'env') [app] adding config value: ('html4_writer', False, 'html') [app] connecting event 'config-inited': [id=2] [app] connecting event 'config-inited': [id=3] [app] connecting event 'config-inited': [id=4] [app] connecting event 'config-inited': [id=5] [app] connecting event 'config-inited': [id=6] [app] connecting event 'config-inited': [id=7] [app] connecting event 'builder-inited': [id=8] [app] connecting event 'html-page-context': [id=9] [app] setting up extension: 'sphinx.ext.mathjax' [app] adding html_math_renderer: mathjax, (, None), (, None) [app] adding config value: ('mathjax_path', 'https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML', 'html') [app] adding config value: ('mathjax_options', {}, 'html') [app] adding config value: ('mathjax_inline', ['\\(', '\\)'], 'html') [app] adding config value: ('mathjax_display', ['\\[', '\\]'], 'html') [app] adding config value: ('mathjax_config', None, 'html') [app] connecting event 'env-updated': [id=10] [app] adding builder: [app] setting up extension: 'sphinx.builders.dummy' [app] adding builder: [app] setting up extension: 'sphinx.builders.gettext' [app] adding builder: [app] adding config value: ('gettext_compact', True, 'gettext') [app] adding config value: ('gettext_location', True, 'gettext') [app] adding config value: ('gettext_uuid', False, 'gettext') [app] adding config value: ('gettext_auto_build', True, 'env') [app] adding config value: ('gettext_additional_targets', [], 'env') [app] adding config value: ('gettext_last_translator', 'FULL NAME ', 'gettext') [app] adding config value: ('gettext_language_team', 'LANGUAGE ', 'gettext') [app] setting up extension: 'sphinx.builders.html' [app] setting up extension: 'sphinx.builders.latex' [app] setting up extension: 'sphinx.builders.latex.transforms' [app] adding transform: [app] adding post transform: [app] adding post transform: [app] adding post transform: [app] adding post transform: [app] adding post transform: [app] adding post transform: [app] adding post transform: [app] adding post transform: [app] adding builder: [app] connecting event 'config-inited': [id=11] [app] adding config value: ('latex_engine', , None, ) [app] adding config value: ('latex_documents', , None) [app] adding config value: ('latex_logo', None, None, []) [app] adding config value: ('latex_appendices', [], None) [app] adding config value: ('latex_use_latex_multicolumn', False, None) [app] adding config value: ('latex_use_xindy', , None, []) [app] adding config value: ('latex_toplevel_sectioning', None, None, ) [app] adding config value: ('latex_domain_indices', True, None, []) [app] adding config value: ('latex_show_urls', 'no', None) [app] adding config value: ('latex_show_pagerefs', False, None) [app] adding config value: ('latex_elements', {}, None) [app] adding config value: ('latex_additional_files', [], None) [app] adding config value: ('latex_docclass', , None) [app] setting up extension: 'sphinx.builders.linkcheck' [app] adding builder: [app] adding config value: ('linkcheck_ignore', [], None) [app] adding config value: ('linkcheck_auth', [], None) [app] adding config value: ('linkcheck_retries', 1, None) [app] adding config value: ('linkcheck_timeout', None, None, []) [app] adding config value: ('linkcheck_workers', 5, None) [app] adding config value: ('linkcheck_anchors', True, None) [app] adding config value: ('linkcheck_anchors_ignore', ['^!'], None) [app] setting up extension: 'sphinx.builders.manpage' [app] adding builder: [app] adding config value: ('man_pages', , None) [app] adding config value: ('man_show_urls', False, None) [app] setting up extension: 'sphinx.builders.singlehtml' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding config value: ('singlehtml_sidebars', . at 0xf5f7f0b8>, 'html') [app] setting up extension: 'sphinx.builders.texinfo' [app] adding builder: [app] adding config value: ('texinfo_documents', , None) [app] adding config value: ('texinfo_appendices', [], None) [app] adding config value: ('texinfo_elements', {}, None) [app] adding config value: ('texinfo_domain_indices', True, None, []) [app] adding config value: ('texinfo_show_urls', 'footnote', None) [app] adding config value: ('texinfo_no_detailmenu', False, None) [app] setting up extension: 'sphinx.builders.text' [app] adding builder: [app] adding config value: ('text_sectionchars', '*=-~"+`', 'env') [app] adding config value: ('text_newlines', 'unix', 'env') [app] adding config value: ('text_add_secnumbers', True, 'env') [app] adding config value: ('text_secnumber_suffix', '. ', 'env') [app] setting up extension: 'sphinx.builders.xml' [app] adding builder: [app] adding builder: [app] adding config value: ('xml_pretty', True, 'env') [app] setting up extension: 'sphinx.config' [app] connecting event 'config-inited': [id=12] [app] connecting event 'config-inited': [id=13] [app] connecting event 'config-inited': [id=14] [app] connecting event 'config-inited': [id=15] [app] connecting event 'config-inited': [id=16] [app] connecting event 'env-get-outdated': [id=17] [app] setting up extension: 'sphinx.domains.c' [app] adding domain: [app] setting up extension: 'sphinx.domains.changeset' [app] adding domain: [app] adding directive: ('deprecated', , None, None, {}) [app] adding directive: ('versionadded', , None, None, {}) [app] adding directive: ('versionchanged', , None, None, {}) [app] setting up extension: 'sphinx.domains.citation' [app] adding domain: [app] adding transform: [app] adding transform: [app] setting up extension: 'sphinx.domains.cpp' [app] adding domain: [app] adding config value: ('cpp_index_common_prefix', [], 'env') [app] adding config value: ('cpp_id_attributes', [], 'env') [app] adding config value: ('cpp_paren_attributes', [], 'env') [app] adding post transform: [app] setting up extension: 'sphinx.domains.index' [app] adding domain: [app] adding directive: ('index', , None, None, {}) [app] adding role: ('index', ) [app] setting up extension: 'sphinx.domains.javascript' [app] adding domain: [app] setting up extension: 'sphinx.domains.math' [app] adding domain: [app] adding role: ('eq', ) [app] setting up extension: 'sphinx.domains.python' [app] adding domain: [app] setting up extension: 'sphinx.domains.rst' [app] adding domain: [app] setting up extension: 'sphinx.domains.std' [app] adding domain: [app] setting up extension: 'sphinx.directives' [app] adding event: 'object-description-transform' [app] setting up extension: 'sphinx.directives.code' [app] setting up extension: 'sphinx.directives.other' [app] setting up extension: 'sphinx.directives.patches' [app] setting up extension: 'sphinx.extension' [app] connecting event 'config-inited': [id=18] [app] setting up extension: 'sphinx.parsers' [app] adding search source_parser: (,) [app] setting up extension: 'sphinx.registry' [app] connecting event 'config-inited': [id=19] [app] setting up extension: 'sphinx.roles' [app] setting up extension: 'sphinx.transforms' [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] setting up extension: 'sphinx.transforms.compact_bullet_list' [app] adding transform: [app] setting up extension: 'sphinx.transforms.i18n' [app] adding transform: [app] adding transform: [app] adding transform: [app] setting up extension: 'sphinx.transforms.references' [app] adding transform: [app] adding transform: [app] adding transform: [app] setting up extension: 'sphinx.transforms.post_transforms' [app] adding post transform: [app] adding post transform: [app] setting up extension: 'sphinx.transforms.post_transforms.code' [app] adding post transform: [app] adding post transform: [app] setting up extension: 'sphinx.transforms.post_transforms.images' [app] adding post transform: [app] adding post transform: [app] setting up extension: 'sphinx.transforms.post_transforms.compat' [app] adding post transform: [app] setting up extension: 'sphinx.util.compat' [app] adding transform: [app] connecting event 'config-inited': [id=20] [app] connecting event 'builder-inited': [id=21] [app] setting up extension: 'sphinx.versioning' [app] adding transform: [app] setting up extension: 'sphinx.environment.collectors.dependencies' [app] adding environment collector: [app] connecting event 'doctree-read': > [id=22] [app] connecting event 'env-merge-info': > [id=23] [app] connecting event 'env-purge-doc': > [id=24] [app] connecting event 'env-get-updated': > [id=25] [app] connecting event 'env-get-outdated': > [id=26] [app] setting up extension: 'sphinx.environment.collectors.asset' [app] adding environment collector: [app] connecting event 'doctree-read': > [id=27] [app] connecting event 'env-merge-info': > [id=28] [app] connecting event 'env-purge-doc': > [id=29] [app] connecting event 'env-get-updated': > [id=30] [app] connecting event 'env-get-outdated': > [id=31] [app] adding environment collector: [app] connecting event 'doctree-read': > [id=32] [app] connecting event 'env-merge-info': > [id=33] [app] connecting event 'env-purge-doc': > [id=34] [app] connecting event 'env-get-updated': > [id=35] [app] connecting event 'env-get-outdated': > [id=36] [app] setting up extension: 'sphinx.environment.collectors.metadata' [app] adding environment collector: [app] connecting event 'doctree-read': > [id=37] [app] connecting event 'env-merge-info': > [id=38] [app] connecting event 'env-purge-doc': > [id=39] [app] connecting event 'env-get-updated': > [id=40] [app] connecting event 'env-get-outdated': > [id=41] [app] setting up extension: 'sphinx.environment.collectors.title' [app] adding environment collector: [app] connecting event 'doctree-read': > [id=42] [app] connecting event 'env-merge-info': > [id=43] [app] connecting event 'env-purge-doc': > [id=44] [app] connecting event 'env-get-updated': > [id=45] [app] connecting event 'env-get-outdated': > [id=46] [app] setting up extension: 'sphinx.environment.collectors.toctree' [app] adding environment collector: [app] connecting event 'doctree-read': > [id=47] [app] connecting event 'env-merge-info': > [id=48] [app] connecting event 'env-purge-doc': > [id=49] [app] connecting event 'env-get-updated': > [id=50] [app] connecting event 'env-get-outdated': > [id=51] [app] setting up extension: 'sphinxcontrib.applehelp' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding config value: ('applehelp_bundle_name', . at 0xf5b18b20>, 'applehelp') [app] adding config value: ('applehelp_bundle_id', None, 'applehelp', []) [app] adding config value: ('applehelp_dev_region', 'en-us', 'applehelp') [app] adding config value: ('applehelp_bundle_version', '1', 'applehelp') [app] adding config value: ('applehelp_icon', None, 'applehelp', []) [app] adding config value: ('applehelp_kb_product', . at 0xf5b27538>, 'applehelp') [app] adding config value: ('applehelp_kb_url', None, 'applehelp', []) [app] adding config value: ('applehelp_remote_url', None, 'applehelp', []) [app] adding config value: ('applehelp_index_anchors', False, 'applehelp', []) [app] adding config value: ('applehelp_min_term_length', None, 'applehelp', []) [app] adding config value: ('applehelp_stopwords', . at 0xf5b274f0>, 'applehelp') [app] adding config value: ('applehelp_locale', . at 0xf5b27ec8>, 'applehelp') [app] adding config value: ('applehelp_title', . at 0xf5b27f10>, 'applehelp') [app] adding config value: ('applehelp_codesign_identity', . at 0xf5b27f58>, 'applehelp') [app] adding config value: ('applehelp_codesign_flags', . at 0xf5b27fa0>, 'applehelp') [app] adding config value: ('applehelp_indexer_path', '/usr/bin/hiutil', 'applehelp') [app] adding config value: ('applehelp_codesign_path', '/usr/bin/codesign', 'applehelp') [app] adding config value: ('applehelp_disable_external_tools', False, 'applehelp') [app] setting up extension: 'sphinxcontrib.devhelp' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding config value: ('devhelp_basename', . at 0xf5b31028>, 'devhelp') [app] setting up extension: 'sphinxcontrib.htmlhelp' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding config value: ('htmlhelp_basename', , None) [app] adding config value: ('htmlhelp_file_suffix', None, 'html', []) [app] adding config value: ('htmlhelp_link_suffix', None, 'html', []) [app] setting up extension: 'sphinxcontrib.serializinghtml' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding builder: [app] setting up extension: 'sphinxcontrib.qthelp' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding config value: ('qthelp_basename', . at 0xf5c58a90>, 'html') [app] adding config value: ('qthelp_namespace', None, 'html', []) [app] adding config value: ('qthelp_theme', 'nonav', 'html') [app] adding config value: ('qthelp_theme_options', {}, 'html') [app] setting up extension: 'alabaster' [app] connecting event 'html-page-context': [id=52] [app] setting up extension: 'sphinx.ext.autodoc' [app] adding autodocumenter: [app] adding directive: ('automodule', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autoclass', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autoexception', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autodata', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autodatadecl', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autofunction', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autodecorator', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('automethod', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autoattribute', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autoproperty', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autoinstanceattribute', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autoslotsattribute', , None, None, {}) [app] adding config value: ('autoclass_content', 'class', True) [app] adding config value: ('autodoc_member_order', 'alphabetic', True) [app] adding config value: ('autodoc_default_flags', [], True) [app] adding config value: ('autodoc_default_options', {}, True) [app] adding config value: ('autodoc_docstring_signature', True, True) [app] adding config value: ('autodoc_mock_imports', [], True) [app] adding config value: ('autodoc_typehints', 'signature', True, ) [app] adding config value: ('autodoc_warningiserror', True, True) [app] adding config value: ('autodoc_inherit_docstrings', True, True) [app] adding event: 'autodoc-before-process-signature' [app] adding event: 'autodoc-process-docstring' [app] adding event: 'autodoc-process-signature' [app] adding event: 'autodoc-skip-member' [app] connecting event 'config-inited': [id=53] [app] setting up extension: 'sphinx.ext.autodoc.type_comment' [app] connecting event 'autodoc-before-process-signature': [id=54] [app] setting up extension: 'sphinx.ext.doctest' [app] adding directive: ('testsetup', , None, None, {}) [app] adding directive: ('testcleanup', , None, None, {}) [app] adding directive: ('doctest', , None, None, {}) [app] adding directive: ('testcode', , None, None, {}) [app] adding directive: ('testoutput', , None, None, {}) [app] adding builder: [app] adding config value: ('doctest_path', [], False) [app] adding config value: ('doctest_test_doctest_blocks', 'default', False) [app] adding config value: ('doctest_global_setup', '', False) [app] adding config value: ('doctest_global_cleanup', '', False) [app] adding config value: ('doctest_default_flags', 41, False) [app] setting up extension: 'sphinx.ext.intersphinx' [app] adding config value: ('intersphinx_mapping', {}, True) [app] adding config value: ('intersphinx_cache_limit', 5, False) [app] adding config value: ('intersphinx_timeout', None, False) [app] connecting event 'config-inited': [id=55] [app] connecting event 'builder-inited': [id=56] [app] connecting event 'missing-reference': [id=57] [app] setting up extension: 'sphinx.ext.todo' [app] adding event: 'todo-defined' [app] adding config value: ('todo_include_todos', False, 'html') [app] adding config value: ('todo_link_only', False, 'html') [app] adding config value: ('todo_emit_warnings', False, 'html') [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {'html': (, ), 'latex': (, ), 'text': (, ), 'man': (, ), 'texinfo': (, )}) [app] adding translation_handlers: , {'html': (, ), 'latex': (, ), 'text': (, ), 'man': (, ), 'texinfo': (, )} [app] adding directive: ('todo', , None, None, {}) [app] adding directive: ('todolist', , None, None, {}) [app] adding domain: [app] connecting event 'doctree-resolved': [id=58] [app] setting up extension: 'sphinx.ext.coverage' [app] adding builder: [app] adding config value: ('coverage_ignore_modules', [], False) [app] adding config value: ('coverage_ignore_functions', [], False) [app] adding config value: ('coverage_ignore_classes', [], False) [app] adding config value: ('coverage_ignore_pyobjects', [], False) [app] adding config value: ('coverage_c_path', [], False) [app] adding config value: ('coverage_c_regexes', {}, False) [app] adding config value: ('coverage_ignore_c_items', {}, False) [app] adding config value: ('coverage_write_headline', True, False) [app] adding config value: ('coverage_skip_undoc_in_source', False, False) [app] setting up extension: 'numpydoc' [app] connecting event 'autodoc-process-docstring': [id=59] [app] connecting event 'autodoc-process-signature': [id=60] [app] adding config value: ('numpydoc_edit_link', None, False) [app] adding config value: ('numpydoc_use_plots', None, False) [app] adding config value: ('numpydoc_show_class_members', True, True) [app] adding config value: ('numpydoc_show_inherited_class_members', True, True) [app] adding config value: ('numpydoc_class_members_toctree', True, True) [app] adding config value: ('numpydoc_citation_re', '[a-z0-9_.-]+', True) [app] adding domain: [app] adding domain: making output directory... done [app] emitting event: 'config-inited'(,) WARNING: html_static_path entry '_static' does not exist [app] emitting event: 'builder-inited'() loading intersphinx inventory from http://docs.python.org/objects.inv... WARNING: failed to reach any of the inventories with the following issues: intersphinx inventory 'http://docs.python.org/objects.inv' not fetchable due to : HTTPConnectionPool(host='docs.python.org', port=80): Max retries exceeded with url: /objects.inv (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')) building [mo]: targets for 0 po files that are out of date building [pickle]: targets for 5 source files that are out of date updating environment: [app] emitting event: 'env-get-outdated'(, {'index', 'tutorial', 'intr [new config] 5 added, 0 changed, 0 removed [app] emitting event: 'env-before-read-docs'(, ['index', 'install', 'intro', 'referenc reading sources... [ 20%] index [app] emitting event: 'env-purge-doc'(, 'index') [app] emitting event: 'source-read'('index', [".. python-blosc documentation master file, created by\n sphinx-quickstart on Sun May [app] emitting event: 'doctree-read'(
,) reading sources... [ 40%] install [app] emitting event: 'env-purge-doc'(, 'install') [app] emitting event: 'source-read'('install', ['------------\nInstallation\n------------\n\npython-blosc comes with C Blosc sources, s [app] emitting event: 'doctree-read'(>,) reading sources... [ 60%] intro [app] emitting event: 'env-purge-doc'(, 'intro') [app] emitting event: 'source-read'('intro', ['------------\nIntroduction\n------------\n\nWhat is python-blosc?\n===================== [app] emitting event: 'doctree-read'(>,) reading sources... [ 80%] reference [app] emitting event: 'env-purge-doc'(, 'reference') [app] emitting event: 'source-read'('reference', ['-----------------\nLibrary Reference\n-----------------\n\nFirst level variables\n== [app] emitting event: 'object-description-transform'('py', 'attribute', >) [app] emitting event: 'object-description-transform'('py', 'attribute', >) [app] emitting event: 'object-description-transform'('py', 'attribute', >) [app] emitting event: 'object-description-transform'('py', 'attribute', >) [app] emitting event: 'object-description-transform'('py', 'attribute', >) [app] emitting event: 'object-description-transform'('py', 'attribute', >) [autodoc] /usr/src/RPM/BUILD/python3-module-blosc-1.5.1/doc/reference.rst:37: input: .. automodule:: blosc :members: compress, compress_ptr, decompress, decompress_ptr, pack_array, unpack_array [autodoc] import blosc [autodoc] import blosc => [app] emitting event: 'autodoc-process-signature'('module', 'blosc', , False, {'members': ['compress', 'compress_ [app] emitting event: 'autodoc-skip-member'('module', 'compress_ptr', , False, {'members': ['compress', 'c [app] emitting event: 'autodoc-skip-member'('module', 'decompress', , False, {'members': ['compress', 'compr [app] emitting event: 'autodoc-skip-member'('module', 'decompress_ptr', , False, {'members': ['compress' [app] emitting event: 'autodoc-skip-member'('module', 'pack_array', , False, {'members': ['compress', 'compr [app] emitting event: 'autodoc-skip-member'('module', 'unpack_array', , False, {'members': ['compress', 'c [autodoc] from blosc import compress [autodoc] import blosc => [autodoc] getattr(_, 'compress') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.compress', , {'members': ['compress', 'compress [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.compress', , {'members': ['compress', 'compress [autodoc] from blosc import compress_ptr [autodoc] import blosc => [autodoc] getattr(_, 'compress_ptr') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.compress_ptr', , {'members': ['compress', ' [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.compress_ptr', , {'members': ['compress', ' [autodoc] from blosc import decompress [autodoc] import blosc => [autodoc] getattr(_, 'decompress') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.decompress', , {'members': ['compress', 'comp [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.decompress', , {'members': ['compress', 'comp [autodoc] from blosc import decompress_ptr [autodoc] import blosc => [autodoc] getattr(_, 'decompress_ptr') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.decompress_ptr', , {'members': ['compress [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.decompress_ptr', , {'members': ['compress [autodoc] from blosc import pack_array [autodoc] import blosc => [autodoc] getattr(_, 'pack_array') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.pack_array', , {'members': ['compress', 'comp [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.pack_array', , {'members': ['compress', 'comp [autodoc] from blosc import unpack_array [autodoc] import blosc => [autodoc] getattr(_, 'unpack_array') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.unpack_array', , {'members': ['compress', ' [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.unpack_array', , {'members': ['compress', ' [autodoc] output: .. py:module:: blosc .. py:function:: compress(bytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]) :module: blosc Compress bytesobj, with a given type size. :Parameters: **bytesobj** : bytes-like object (supporting the buffer interface) The data to be compressed. **typesize** : int The data type size. **clevel** : int (optional) The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. **shuffle** : int (optional) The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. **cname** : string (optional) The name of the compressor used internally in Blosc. It can be any of the supported by Blosc ('blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd' and maybe others too). The default is 'blosclz'. :Returns: **out** : str / bytes The compressed data in form of a Python str / bytes object. :Raises: **TypeError** If bytesobj doesn't support the buffer interface. **ValueError** If bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. .. rubric:: Examples >>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) True .. py:function:: compress_ptr(address, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]) :module: blosc Compress the data at address with given items and typesize. :Parameters: **address** : int or long the pointer to the data to be compressed **items** : int The number of items (of typesize) to be compressed. **typesize** : int The data type size. **clevel** : int (optional) The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. **shuffle** : int (optional) The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. **cname** : string (optional) The name of the compressor used internally in Blosc. It can be any of the supported by Blosc ('blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd' and maybe others too). The default is 'blosclz'. :Returns: **out** : str / bytes The compressed data in form of a Python str / bytes object. :Raises: **TypeError** If address is not of type int or long. **ValueError** If items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. .. rubric:: Notes This function can be used anywhere that a memory address is available in Python. For example the Numpy "__array_interface__['data'][0]" construct, or when using the ctypes modules. Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. .. rubric:: Examples >>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True >>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True .. py:function:: decompress(bytesobj) :module: blosc Decompresses a bytesobj compressed object. :Parameters: **bytesobj** : str / bytes The data to be decompressed. **as_bytearray** : bool, optional If this flag is True then the return type will be a bytearray object instead of a bytesobject. :Returns: **out** : str / bytes or bytearray The decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. :Raises: **TypeError** If bytesobj is not of type bytes or string. .. rubric:: Examples >>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray True .. py:function:: decompress_ptr(bytesobj, address) :module: blosc Decompresses a bytesobj compressed object into the memory at address. :Parameters: **bytesobj** : str / bytes The data to be decompressed. **address** : int or long the pointer to the data to be compressed :Returns: **nbytes** : int the number of bytes written to the buffer :Raises: **TypeError** If bytesobj is not of type bytes or string. If address is not of type int or long. .. rubric:: Notes This function can be used anywhere that a memory address is available in Python. For example the Numpy "__array_interface__['data'][0]" construct, or when using the ctypes modules. Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. .. rubric:: Examples >>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True >>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize True .. py:function:: pack_array(array[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]) :module: blosc Pack (compress) a NumPy array. :Parameters: **array** : ndarray The NumPy array to be packed. **clevel** : int (optional) The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. **shuffle** : int (optional) The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. **cname** : string (optional) The name of the compressor used internally in Blosc. It can be any of the supported by Blosc ('blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd' and maybe others too). The default is 'blosclz'. :Returns: **out** : str / bytes The packed array in form of a Python str / bytes object. :Raises: **TypeError** If array does not quack like a numpy ndarray. **ValueError** If array.itemsize * array.size is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. .. rubric:: Examples >>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True .. py:function:: unpack_array(packed_array) :module: blosc Unpack (decompress) a packed NumPy array. :Parameters: **packed_array** : str / bytes The packed array to be decompressed. :Returns: **out** : ndarray The decompressed data in form of a NumPy array. :Raises: **TypeError** If packed_array is not of type bytes or string. .. rubric:: Examples >>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True [app] emitting event: 'object-description-transform'('py', 'function', >) [app] emitting event: 'object-description-transform'('py', 'function',

) [app] emitting event: 'object-description-transform'('py', 'function', >) [app] emitting event: 'object-description-transform'('py', 'function',

) [app] emitting event: 'object-description-transform'('py', 'function', >) [app] emitting event: 'object-description-transform'('py', 'function', >) [autodoc] /usr/src/RPM/BUILD/python3-module-blosc-1.5.1/doc/reference.rst:44: input: .. automodule:: blosc :members: clib_info, compressor_list, detect_number_of_cores, free_resources, get_clib, set_blocksize, set_nthreads, set_releasegil, print_versions, test [autodoc] import blosc [autodoc] import blosc => [app] emitting event: 'autodoc-process-signature'('module', 'blosc', , False, {'members': ['clib_info', 'compre [app] emitting event: 'autodoc-skip-member'('module', 'compressor_list', , False, {'members': ['clib_in [app] emitting event: 'autodoc-skip-member'('module', 'detect_number_of_cores', , False, {'membe [app] emitting event: 'autodoc-skip-member'('module', 'free_resources', , False, {'members': ['clib_info [app] emitting event: 'autodoc-skip-member'('module', 'get_clib', , False, {'members': ['clib_info', 'compress [app] emitting event: 'autodoc-skip-member'('module', 'set_blocksize', , False, {'members': ['clib_info', [app] emitting event: 'autodoc-skip-member'('module', 'set_nthreads', , False, {'members': ['clib_info', ' [app] emitting event: 'autodoc-skip-member'('module', 'set_releasegil', , False, {'members': ['clib_info [app] emitting event: 'autodoc-skip-member'('module', 'print_versions', , False, {'members': ['clib_info [app] emitting event: 'autodoc-skip-member'('module', 'test', , True, {'members': ['clib_info', 'compressor_list', [autodoc] from blosc import clib_info [autodoc] import blosc => [autodoc] getattr(_, 'clib_info') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.clib_info', , {'members': ['clib_info', 'compr [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.clib_info', , {'members': ['clib_info', 'compr [autodoc] from blosc import compressor_list [autodoc] import blosc => [autodoc] getattr(_, 'compressor_list') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.compressor_list', , {'members': ['clib_i [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.compressor_list', , {'members': ['clib_i [autodoc] from blosc import detect_number_of_cores [autodoc] import blosc => [autodoc] getattr(_, 'detect_number_of_cores') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.detect_number_of_cores', , {'memb [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.detect_number_of_cores', , {'memb [autodoc] from blosc import free_resources [autodoc] import blosc => [autodoc] getattr(_, 'free_resources') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.free_resources', , {'members': ['clib_inf [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.free_resources', , {'members': ['clib_inf [autodoc] from blosc import get_clib [autodoc] import blosc => [autodoc] getattr(_, 'get_clib') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.get_clib', , {'members': ['clib_info', 'compres [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.get_clib', , {'members': ['clib_info', 'compres [autodoc] from blosc import set_blocksize [autodoc] import blosc => [autodoc] getattr(_, 'set_blocksize') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.set_blocksize', , {'members': ['clib_info' [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.set_blocksize', , {'members': ['clib_info' [autodoc] from blosc import set_nthreads [autodoc] import blosc => [autodoc] getattr(_, 'set_nthreads') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.set_nthreads', , {'members': ['clib_info', [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.set_nthreads', , {'members': ['clib_info', [autodoc] from blosc import set_releasegil [autodoc] import blosc => [autodoc] getattr(_, 'set_releasegil') [autodoc] => [app] emitting event: 'autodoc-process-signature'('function', 'blosc.set_releasegil', , {'members': ['clib_inf [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.set_releasegil', , {'members': ['clib_inf [autodoc] from blosc import print_versions [autodoc] import blosc => [autodoc] getattr(_, 'print_versions') [autodoc] => [app] emitting event: 'autodoc-before-process-signature'(, False) [app] emitting event: 'autodoc-process-signature'('function', 'blosc.print_versions', , {'members': ['clib_inf [app] emitting event: 'autodoc-process-docstring'('function', 'blosc.print_versions', , {'members': ['clib_inf [autodoc] output: .. py:module:: blosc .. py:function:: clib_info(cname) :module: blosc Return info for compression libraries in C library. :Parameters: **cname** : str The compressor name. :Returns: **out** : tuple The associated library name and version. .. py:function:: compressor_list() :module: blosc Returns a list of compressors available in C library. :Parameters: **None** :Returns: **out** : list The list of names. .. py:function:: detect_number_of_cores() :module: blosc Detect the number of cores in this system. :Returns: **out** : int The number of cores in this system. .. py:function:: free_resources() :module: blosc Free possible memory temporaries and thread resources. :Returns: **out** : None .. rubric:: Notes Blosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while. .. rubric:: Examples >>> blosc.free_resources() >>> .. py:function:: get_clib(bytesobj) :module: blosc Return the name of the compression library for Blosc `bytesobj` buffer. :Parameters: **bytesobj** : str / bytes The compressed buffer. :Returns: **out** : str The name of the compression library. .. py:function:: set_blocksize(blocksize) :module: blosc Force the use of a specific blocksize. If 0, an automatic blocksize will be used (the default). .. rubric:: Notes This is a low-level function and is recommened for expert users only. .. rubric:: Examples >>> blosc.set_blocksize(512) .. py:function:: set_nthreads(nthreads) :module: blosc Set the number of threads to be used during Blosc operation. :Parameters: **nthreads** : int The number of threads to be used during Blosc operation. :Returns: **out** : int The previous number of used threads. :Raises: **ValueError** If nthreads is larger that the maximum number of threads blosc can use. .. rubric:: Notes The number of threads for Blosc is the maximum number of cores detected on your machine (via `detect_number_of_cores`). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores. .. rubric:: Examples Set the number of threads to 2 and then to 1: >>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2 .. py:function:: set_releasegil( gitstate ) :module: blosc Sets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False. .. rubric:: Notes Designed to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes. .. rubric:: Examples >>> oldReleaseState = blosc.set_releasegil(True) .. py:function:: print_versions() :module: blosc Print all the versions of software that python-blosc relies on. [app] emitting event: 'object-description-transform'('py', 'function', >) [app] emitting event: 'object-description-transform'('py', 'function', >) [app] emitting event: 'object-description-transform'('py', 'function', >) [app] emitting event: 'object-description-transform'('py', 'function', ) [app] emitting event: 'object-description-transform'('py', 'function', >) [app] emitting event: 'object-description-transform'('py', 'function', ) [app] emitting event: 'object-description-transform'('py', 'function', ) [app] emitting event: 'object-description-transform'('py', 'function', ) [app] emitting event: 'object-description-transform'('py', 'function', >) [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Parameters [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Returns [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Raises [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Parameters [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Returns [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Raises [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Parameters [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Returns [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Raises [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Parameters [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Returns [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Raises [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Parameters [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Returns [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Raises [i18n] PATCH: 'docutils.nodes.term' to have rawsource: If array.itemsize * array.size is larger than the maximum allowed [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Parameters [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Returns [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Raises [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Parameters [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Returns [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Parameters [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Returns [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Returns [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Returns [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Parameters [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Returns [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Parameters [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Returns [i18n] PATCH: 'docutils.nodes.field_name' to have source and line: Raises [app] emitting event: 'doctree-read'(>,) /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/__init__.py:docstring of blosc.pack_array:47: (WARNING/2) Definition list ends without a blank line; unexpected unindent. [filtered system message] reading sources... [100%] tutorial [app] emitting event: 'env-purge-doc'(, 'tutorial') [app] emitting event: 'source-read'('tutorial', ['---------\nTutorials\n---------\n\nUsing `python-blosc` (or just `blosc`, because we [app] emitting event: 'doctree-read'(>,) [app] emitting event: 'env-updated'(,) /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/__init__.py:docstring of blosc.pack_array:47: WARNING: Definition list ends without a blank line; unexpected unindent. /usr/src/RPM/BUILD/python3-module-blosc-1.5.1/doc/reference.rst:2: WARNING: duplicate object description of blosc, other instance in reference, use :noindex: for one of them /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/__init__.py:docstring of blosc.print_versions:17: WARNING: Duplicate ID: "module-blosc". looking for now-outdated files... [app] emitting event: 'env-get-updated'(,) none found pickling environment... done checking consistency... [app] emitting event: 'env-check-consistency'(,) done docnames to write: index, install, intro, reference, tutorial preparing documents... done writing output... [ 20%] index [app] emitting event: 'doctree-resolved'(

, 'index') SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for python-blosc documentation master file, created by sphinx-quickstart on Sun May 5 19:38:24 2013. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive.
Welcome to python-blosc’s documentation!Contents:IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities
Indices and tablesIndexModule IndexSearch Page
SphinxTranslator.dispatch_visit calling visit_comment for python-blosc documentation master file, created by sphinx-quickstart on Sun May 5 19:38:24 2013. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. SphinxTranslator.dispatch_visit calling visit_section for
Welcome to python-blosc’s documentation!Contents:IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities
SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_paragraph for Contents: SphinxTranslator.dispatch_visit calling visit_Text for Contents: SphinxTranslator.dispatch_departure calling depart_Text for Contents: SphinxTranslator.dispatch_departure calling depart_paragraph for Contents: SphinxTranslator.dispatch_visit calling visit_compound for IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_compact_paragraph for IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_bullet_list for IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_list_item for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Introduction SphinxTranslator.dispatch_visit calling visit_reference for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_reference for Introduction SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Introduction SphinxTranslator.dispatch_visit calling visit_bullet_list for What is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_list_item for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_compact_paragraph for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_reference for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_Text for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_Text for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_reference for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_compact_paragraph for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_list_item for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_list_item for Mailing list SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Mailing list SphinxTranslator.dispatch_visit calling visit_reference for Mailing list SphinxTranslator.dispatch_visit calling visit_Text for Mailing list SphinxTranslator.dispatch_departure calling depart_Text for Mailing list SphinxTranslator.dispatch_departure calling depart_reference for Mailing list SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Mailing list SphinxTranslator.dispatch_departure calling depart_list_item for Mailing list SphinxTranslator.dispatch_departure calling depart_bullet_list for What is python-blosc?Mailing list SphinxTranslator.dispatch_departure calling depart_list_item for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_list_item for InstallationInstalling from PyPI repositoryBuilding manuallyTestingInstalling SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Installation SphinxTranslator.dispatch_visit calling visit_reference for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_reference for Installation SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Installation SphinxTranslator.dispatch_visit calling visit_bullet_list for Installing from PyPI repositoryBuilding manuallyTestingInstalling SphinxTranslator.dispatch_visit calling visit_list_item for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_reference for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_Text for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_Text for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_reference for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_list_item for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_list_item for Building manually SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Building manually SphinxTranslator.dispatch_visit calling visit_reference for Building manually SphinxTranslator.dispatch_visit calling visit_Text for Building manually SphinxTranslator.dispatch_departure calling depart_Text for Building manually SphinxTranslator.dispatch_departure calling depart_reference for Building manually SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Building manually SphinxTranslator.dispatch_departure calling depart_list_item for Building manually SphinxTranslator.dispatch_visit calling visit_list_item for Testing SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Testing SphinxTranslator.dispatch_visit calling visit_reference for Testing SphinxTranslator.dispatch_visit calling visit_Text for Testing SphinxTranslator.dispatch_departure calling depart_Text for Testing SphinxTranslator.dispatch_departure calling depart_reference for Testing SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Testing SphinxTranslator.dispatch_departure calling depart_list_item for Testing SphinxTranslator.dispatch_visit calling visit_list_item for Installing SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Installing SphinxTranslator.dispatch_visit calling visit_reference for Installing SphinxTranslator.dispatch_visit calling visit_Text for Installing SphinxTranslator.dispatch_departure calling depart_Text for Installing SphinxTranslator.dispatch_departure calling depart_reference for Installing SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Installing SphinxTranslator.dispatch_departure calling depart_list_item for Installing SphinxTranslator.dispatch_departure calling depart_bullet_list for Installing from PyPI repositoryBuilding manuallyTestingInstalling SphinxTranslator.dispatch_departure calling depart_list_item for InstallationInstalling from PyPI repositoryBuilding manuallyTestingInstalling SphinxTranslator.dispatch_visit calling visit_list_item for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Tutorials SphinxTranslator.dispatch_visit calling visit_reference for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_reference for Tutorials SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Tutorials SphinxTranslator.dispatch_visit calling visit_bullet_list for Compressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_list_item for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_reference for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_Text for Compressing and decompressing with SphinxTranslator.dispatch_departure calling depart_Text for Compressing and decompressing with SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_departure calling depart_reference for Compressing and decompressing with blosc SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Compressing and decompressing with blosc SphinxTranslator.dispatch_departure calling depart_list_item for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_list_item for Using different filters SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Using different filters SphinxTranslator.dispatch_visit calling visit_reference for Using different filters SphinxTranslator.dispatch_visit calling visit_Text for Using different filters SphinxTranslator.dispatch_departure calling depart_Text for Using different filters SphinxTranslator.dispatch_departure calling depart_reference for Using different filters SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Using different filters SphinxTranslator.dispatch_departure calling depart_list_item for Using different filters SphinxTranslator.dispatch_visit calling visit_list_item for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_reference for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_Text for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_Text for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_reference for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_list_item for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_list_item for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_reference for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_Text for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_Text for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_reference for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_list_item for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_list_item for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_reference for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_Text for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_Text for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_reference for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_list_item for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_list_item for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_reference for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_Text for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_Text for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_reference for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_list_item for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_list_item for Links to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Links to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_reference for Links to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_Text for Links to external discussions on SphinxTranslator.dispatch_departure calling depart_Text for Links to external discussions on SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for optimization SphinxTranslator.dispatch_departure calling depart_Text for optimization SphinxTranslator.dispatch_departure calling depart_reference for Links to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Links to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_list_item for Links to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_bullet_list for Compressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_list_item for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_list_item for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Library Reference SphinxTranslator.dispatch_visit calling visit_reference for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_reference for Library Reference SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Library Reference SphinxTranslator.dispatch_visit calling visit_bullet_list for First level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_list_item for First level variables SphinxTranslator.dispatch_visit calling visit_compact_paragraph for First level variables SphinxTranslator.dispatch_visit calling visit_reference for First level variables SphinxTranslator.dispatch_visit calling visit_Text for First level variables SphinxTranslator.dispatch_departure calling depart_Text for First level variables SphinxTranslator.dispatch_departure calling depart_reference for First level variables SphinxTranslator.dispatch_departure calling depart_compact_paragraph for First level variables SphinxTranslator.dispatch_departure calling depart_list_item for First level variables SphinxTranslator.dispatch_visit calling visit_list_item for Public functions SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Public functions SphinxTranslator.dispatch_visit calling visit_reference for Public functions SphinxTranslator.dispatch_visit calling visit_Text for Public functions SphinxTranslator.dispatch_departure calling depart_Text for Public functions SphinxTranslator.dispatch_departure calling depart_reference for Public functions SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Public functions SphinxTranslator.dispatch_departure calling depart_list_item for Public functions SphinxTranslator.dispatch_visit calling visit_list_item for Utilities SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Utilities SphinxTranslator.dispatch_visit calling visit_reference for Utilities SphinxTranslator.dispatch_visit calling visit_Text for Utilities SphinxTranslator.dispatch_departure calling depart_Text for Utilities SphinxTranslator.dispatch_departure calling depart_reference for Utilities SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Utilities SphinxTranslator.dispatch_departure calling depart_list_item for Utilities SphinxTranslator.dispatch_departure calling depart_bullet_list for First level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_list_item for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_bullet_list for IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_compact_paragraph for IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_compound for IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_section for
Welcome to python-blosc’s documentation!Contents:IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities
SphinxTranslator.dispatch_visit calling visit_section for
Indices and tablesIndexModule IndexSearch Page
SphinxTranslator.dispatch_visit calling visit_title for Indices and tables SphinxTranslator.dispatch_visit calling visit_Text for Indices and tables SphinxTranslator.dispatch_departure calling depart_Text for Indices and tables SphinxTranslator.dispatch_departure calling depart_title for Indices and tables SphinxTranslator.dispatch_visit calling visit_bullet_list for IndexModule IndexSearch Page SphinxTranslator.dispatch_visit calling visit_list_item for Index SphinxTranslator.dispatch_visit calling visit_paragraph for Index SphinxTranslator.dispatch_visit calling visit_reference for Index SphinxTranslator.dispatch_visit calling visit_inline for Index SphinxTranslator.dispatch_visit calling visit_Text for Index SphinxTranslator.dispatch_departure calling depart_Text for Index SphinxTranslator.dispatch_departure calling depart_inline for Index SphinxTranslator.dispatch_departure calling depart_reference for Index SphinxTranslator.dispatch_departure calling depart_paragraph for Index SphinxTranslator.dispatch_departure calling depart_list_item for Index SphinxTranslator.dispatch_visit calling visit_list_item for Module Index SphinxTranslator.dispatch_visit calling visit_paragraph for Module Index SphinxTranslator.dispatch_visit calling visit_reference for Module Index SphinxTranslator.dispatch_visit calling visit_inline for Module Index SphinxTranslator.dispatch_visit calling visit_Text for Module Index SphinxTranslator.dispatch_departure calling depart_Text for Module Index SphinxTranslator.dispatch_departure calling depart_inline for Module Index SphinxTranslator.dispatch_departure calling depart_reference for Module Index SphinxTranslator.dispatch_departure calling depart_paragraph for Module Index SphinxTranslator.dispatch_departure calling depart_list_item for Module Index SphinxTranslator.dispatch_visit calling visit_list_item for Search Page SphinxTranslator.dispatch_visit calling visit_paragraph for Search Page SphinxTranslator.dispatch_visit calling visit_reference for Search Page SphinxTranslator.dispatch_visit calling visit_inline for Search Page SphinxTranslator.dispatch_visit calling visit_Text for Search Page SphinxTranslator.dispatch_departure calling depart_Text for Search Page SphinxTranslator.dispatch_departure calling depart_inline for Search Page SphinxTranslator.dispatch_departure calling depart_reference for Search Page SphinxTranslator.dispatch_departure calling depart_paragraph for Search Page SphinxTranslator.dispatch_departure calling depart_list_item for Search Page SphinxTranslator.dispatch_departure calling depart_bullet_list for IndexModule IndexSearch Page SphinxTranslator.dispatch_departure calling depart_section for
Indices and tablesIndexModule IndexSearch Page
SphinxTranslator.dispatch_departure calling depart_document for python-blosc documentation master file, created by sphinx-quickstart on Sun May 5 19:38:24 2013. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive.
Welcome to python-blosc’s documentation!Contents:IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities
Indices and tablesIndexModule IndexSearch Page
SphinxTranslator.dispatch_visit calling visit_document for Introduction SphinxTranslator.dispatch_visit calling visit_title for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_title for Introduction SphinxTranslator.dispatch_departure calling depart_document for Introduction SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation!Indices and tables SphinxTranslator.dispatch_visit calling visit_bullet_list for Welcome to python-blosc’s documentation!Indices and tables SphinxTranslator.dispatch_visit calling visit_list_item for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_reference for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_reference for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_bullet_list for SphinxTranslator.dispatch_departure calling depart_list_item for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_list_item for Indices and tables SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Indices and tables SphinxTranslator.dispatch_visit calling visit_reference for Indices and tables SphinxTranslator.dispatch_visit calling visit_Text for Indices and tables SphinxTranslator.dispatch_departure calling depart_Text for Indices and tables SphinxTranslator.dispatch_departure calling depart_reference for Indices and tables SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Indices and tables SphinxTranslator.dispatch_departure calling depart_list_item for Indices and tables SphinxTranslator.dispatch_departure calling depart_bullet_list for Welcome to python-blosc’s documentation!Indices and tables SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation!Indices and tables [app] emitting event: 'html-page-context'('index', 'page.html', {'parents': [], 'prev': None, 'next': {'link': 'intro/', 'title': 'Introducti writing output... [ 40%] install [app] emitting event: 'doctree-resolved'(>, 'install') SphinxTranslator.dispatch_visit calling visit_document for Installation SphinxTranslator.dispatch_visit calling visit_title for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_title for Installation SphinxTranslator.dispatch_departure calling depart_document for Installation SphinxTranslator.dispatch_visit calling visit_document for
Installationpython-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested).Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too.This package supports Python 2.6, 2.7 and 3.3 or higher versions.
Installing from PyPI repositoryDo:$ pip install -U blosc
Building manuallyFirst, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources.Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not.
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
TestingAfter compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py:$ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode)Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite:$ nosetests --with-doctest (add -v for verbose mode)Once installed, you can re-run the tests at any time with:$ python -c "import blosc; blosc.test()"
InstallingInstall it as a typical Python package:$ python setup.py install
SphinxTranslator.dispatch_visit calling visit_section for
Installationpython-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested).Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too.This package supports Python 2.6, 2.7 and 3.3 or higher versions.
Installing from PyPI repositoryDo:$ pip install -U blosc
Building manuallyFirst, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources.Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not.
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
TestingAfter compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py:$ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode)Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite:$ nosetests --with-doctest (add -v for verbose mode)Once installed, you can re-run the tests at any time with:$ python -c "import blosc; blosc.test()"
InstallingInstall it as a typical Python package:$ python setup.py install
SphinxTranslator.dispatch_visit calling visit_title for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_title for Installation SphinxTranslator.dispatch_visit calling visit_paragraph for python-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested). SphinxTranslator.dispatch_visit calling visit_Text for python-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested). SphinxTranslator.dispatch_departure calling depart_Text for python-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested). SphinxTranslator.dispatch_departure calling depart_paragraph for python-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested). SphinxTranslator.dispatch_visit calling visit_paragraph for Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too. SphinxTranslator.dispatch_visit calling visit_Text for Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too. SphinxTranslator.dispatch_departure calling depart_Text for Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too. SphinxTranslator.dispatch_departure calling depart_paragraph for Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too. SphinxTranslator.dispatch_visit calling visit_paragraph for This package supports Python 2.6, 2.7 and 3.3 or higher versions. SphinxTranslator.dispatch_visit calling visit_Text for This package supports Python 2.6, 2.7 and 3.3 or higher versions. SphinxTranslator.dispatch_departure calling depart_Text for This package supports Python 2.6, 2.7 and 3.3 or higher versions. SphinxTranslator.dispatch_departure calling depart_paragraph for This package supports Python 2.6, 2.7 and 3.3 or higher versions. SphinxTranslator.dispatch_visit calling visit_section for
Installing from PyPI repositoryDo:$ pip install -U blosc
SphinxTranslator.dispatch_visit calling visit_title for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_Text for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_Text for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_title for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_paragraph for Do: SphinxTranslator.dispatch_visit calling visit_Text for Do: SphinxTranslator.dispatch_departure calling depart_Text for Do: SphinxTranslator.dispatch_departure calling depart_paragraph for Do: SphinxTranslator.dispatch_visit calling visit_literal_block for $ pip install -U blosc SphinxTranslator.dispatch_departure calling depart_section for
Installing from PyPI repositoryDo:$ pip install -U blosc
SphinxTranslator.dispatch_visit calling visit_section for
Building manuallyFirst, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources.Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not.
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
SphinxTranslator.dispatch_visit calling visit_title for Building manually SphinxTranslator.dispatch_visit calling visit_Text for Building manually SphinxTranslator.dispatch_departure calling depart_Text for Building manually SphinxTranslator.dispatch_departure calling depart_title for Building manually SphinxTranslator.dispatch_visit calling visit_paragraph for First, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources. SphinxTranslator.dispatch_visit calling visit_Text for First, go to the python-blosc official repository at SphinxTranslator.dispatch_departure calling depart_Text for First, go to the python-blosc official repository at SphinxTranslator.dispatch_visit calling visit_reference for https://github.com/Blosc/python-blosc SphinxTranslator.dispatch_visit calling visit_Text for https://github.com/Blosc/python-blosc SphinxTranslator.dispatch_departure calling depart_Text for https://github.com/Blosc/python-blosc SphinxTranslator.dispatch_departure calling depart_reference for https://github.com/Blosc/python-blosc SphinxTranslator.dispatch_visit calling visit_Text for and download the sources. SphinxTranslator.dispatch_departure calling depart_Text for and download the sources. SphinxTranslator.dispatch_departure calling depart_paragraph for First, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources. SphinxTranslator.dispatch_visit calling visit_paragraph for Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not. SphinxTranslator.dispatch_visit calling visit_Text for Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not. SphinxTranslator.dispatch_departure calling depart_Text for Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not. SphinxTranslator.dispatch_departure calling depart_paragraph for Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not. SphinxTranslator.dispatch_visit calling visit_section for
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
SphinxTranslator.dispatch_visit calling visit_title for Compiling without an installed Blosc library SphinxTranslator.dispatch_visit calling visit_Text for Compiling without an installed Blosc library SphinxTranslator.dispatch_departure calling depart_Text for Compiling without an installed Blosc library SphinxTranslator.dispatch_departure calling depart_title for Compiling without an installed Blosc library SphinxTranslator.dispatch_visit calling visit_paragraph for python-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do: SphinxTranslator.dispatch_visit calling visit_Text for python-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do: SphinxTranslator.dispatch_departure calling depart_Text for python-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do: SphinxTranslator.dispatch_departure calling depart_paragraph for python-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do: SphinxTranslator.dispatch_visit calling visit_literal_block for $ python setup.py build_ext --inplace SphinxTranslator.dispatch_visit calling visit_paragraph for That’s all. You can proceed with testing section now. SphinxTranslator.dispatch_visit calling visit_Text for That’s all. You can proceed with testing section now. SphinxTranslator.dispatch_departure calling depart_Text for That’s all. You can proceed with testing section now. SphinxTranslator.dispatch_departure calling depart_paragraph for That’s all. You can proceed with testing section now. SphinxTranslator.dispatch_departure calling depart_section for
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
SphinxTranslator.dispatch_visit calling visit_section for
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
SphinxTranslator.dispatch_visit calling visit_title for Compiling with an installed Blosc library SphinxTranslator.dispatch_visit calling visit_Text for Compiling with an installed Blosc library SphinxTranslator.dispatch_departure calling depart_Text for Compiling with an installed Blosc library SphinxTranslator.dispatch_departure calling depart_title for Compiling with an installed Blosc library SphinxTranslator.dispatch_visit calling visit_paragraph for In case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways. SphinxTranslator.dispatch_visit calling visit_Text for In case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways. SphinxTranslator.dispatch_departure calling depart_Text for In case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways. SphinxTranslator.dispatch_departure calling depart_paragraph for In case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways. SphinxTranslator.dispatch_visit calling visit_paragraph for Using an environment variable: SphinxTranslator.dispatch_visit calling visit_Text for Using an environment variable: SphinxTranslator.dispatch_departure calling depart_Text for Using an environment variable: SphinxTranslator.dispatch_departure calling depart_paragraph for Using an environment variable: SphinxTranslator.dispatch_visit calling visit_literal_block for $ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplace SphinxTranslator.dispatch_visit calling visit_paragraph for Using a flag: SphinxTranslator.dispatch_visit calling visit_Text for Using a flag: SphinxTranslator.dispatch_departure calling depart_Text for Using a flag: SphinxTranslator.dispatch_departure calling depart_paragraph for Using a flag: SphinxTranslator.dispatch_visit calling visit_literal_block for $ python setup.py build_ext --inplace --blosc=/usr/local SphinxTranslator.dispatch_departure calling depart_section for
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
SphinxTranslator.dispatch_visit calling visit_section for
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
SphinxTranslator.dispatch_visit calling visit_title for Generating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_Text for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_Text for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_title for Generating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_paragraph for In case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do: SphinxTranslator.dispatch_visit calling visit_Text for In case you want to generate the documentation locally, you will need to have the SphinxTranslator.dispatch_departure calling depart_Text for In case you want to generate the documentation locally, you will need to have the SphinxTranslator.dispatch_visit calling visit_literal for Sphinx SphinxTranslator.dispatch_visit calling visit_Text for Sphinx SphinxTranslator.dispatch_departure calling depart_Text for Sphinx SphinxTranslator.dispatch_departure calling depart_literal for Sphinx SphinxTranslator.dispatch_visit calling visit_Text for documentation system, as well as the SphinxTranslator.dispatch_departure calling depart_Text for documentation system, as well as the SphinxTranslator.dispatch_visit calling visit_literal for numpydoc SphinxTranslator.dispatch_visit calling visit_Text for numpydoc SphinxTranslator.dispatch_departure calling depart_Text for numpydoc SphinxTranslator.dispatch_departure calling depart_literal for numpydoc SphinxTranslator.dispatch_visit calling visit_Text for extension, installed. Then go down to SphinxTranslator.dispatch_departure calling depart_Text for extension, installed. Then go down to SphinxTranslator.dispatch_visit calling visit_literal for doc/ SphinxTranslator.dispatch_visit calling visit_Text for doc/ SphinxTranslator.dispatch_departure calling depart_Text for doc/ SphinxTranslator.dispatch_departure calling depart_literal for doc/ SphinxTranslator.dispatch_visit calling visit_Text for directory and do: SphinxTranslator.dispatch_departure calling depart_Text for directory and do: SphinxTranslator.dispatch_departure calling depart_paragraph for In case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do: SphinxTranslator.dispatch_visit calling visit_literal_block for $ make html|latex|latexpdf SphinxTranslator.dispatch_departure calling depart_section for
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
SphinxTranslator.dispatch_departure calling depart_section for
Building manuallyFirst, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources.Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not.
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
SphinxTranslator.dispatch_visit calling visit_section for
TestingAfter compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py:$ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode)Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite:$ nosetests --with-doctest (add -v for verbose mode)Once installed, you can re-run the tests at any time with:$ python -c "import blosc; blosc.test()"
SphinxTranslator.dispatch_visit calling visit_title for Testing SphinxTranslator.dispatch_visit calling visit_Text for Testing SphinxTranslator.dispatch_departure calling depart_Text for Testing SphinxTranslator.dispatch_departure calling depart_title for Testing SphinxTranslator.dispatch_visit calling visit_paragraph for After compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py: SphinxTranslator.dispatch_visit calling visit_Text for After compiling, you can quickly check that the package is sane by running the doctests in SphinxTranslator.dispatch_departure calling depart_Text for After compiling, you can quickly check that the package is sane by running the doctests in SphinxTranslator.dispatch_visit calling visit_literal for blosc/test.py SphinxTranslator.dispatch_visit calling visit_Text for blosc/test.py SphinxTranslator.dispatch_departure calling depart_Text for blosc/test.py SphinxTranslator.dispatch_departure calling depart_literal for blosc/test.py SphinxTranslator.dispatch_visit calling visit_Text for : SphinxTranslator.dispatch_departure calling depart_Text for : SphinxTranslator.dispatch_departure calling depart_paragraph for After compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py: SphinxTranslator.dispatch_visit calling visit_literal_block for $ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode) SphinxTranslator.dispatch_visit calling visit_paragraph for Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite: SphinxTranslator.dispatch_visit calling visit_Text for Or alternatively, you can use the third-party SphinxTranslator.dispatch_departure calling depart_Text for Or alternatively, you can use the third-party SphinxTranslator.dispatch_visit calling visit_literal for nosetests SphinxTranslator.dispatch_visit calling visit_Text for nosetests SphinxTranslator.dispatch_departure calling depart_Text for nosetests SphinxTranslator.dispatch_departure calling depart_literal for nosetests SphinxTranslator.dispatch_visit calling visit_Text for script to run both the doctests and the test suite: SphinxTranslator.dispatch_departure calling depart_Text for script to run both the doctests and the test suite: SphinxTranslator.dispatch_departure calling depart_paragraph for Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite: SphinxTranslator.dispatch_visit calling visit_literal_block for $ nosetests --with-doctest (add -v for verbose mode) SphinxTranslator.dispatch_visit calling visit_paragraph for Once installed, you can re-run the tests at any time with: SphinxTranslator.dispatch_visit calling visit_Text for Once installed, you can re-run the tests at any time with: SphinxTranslator.dispatch_departure calling depart_Text for Once installed, you can re-run the tests at any time with: SphinxTranslator.dispatch_departure calling depart_paragraph for Once installed, you can re-run the tests at any time with: SphinxTranslator.dispatch_visit calling visit_literal_block for $ python -c "import blosc; blosc.test()" SphinxTranslator.dispatch_departure calling depart_section for
TestingAfter compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py:$ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode)Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite:$ nosetests --with-doctest (add -v for verbose mode)Once installed, you can re-run the tests at any time with:$ python -c "import blosc; blosc.test()"
SphinxTranslator.dispatch_visit calling visit_section for
InstallingInstall it as a typical Python package:$ python setup.py install
SphinxTranslator.dispatch_visit calling visit_title for Installing SphinxTranslator.dispatch_visit calling visit_Text for Installing SphinxTranslator.dispatch_departure calling depart_Text for Installing SphinxTranslator.dispatch_departure calling depart_title for Installing SphinxTranslator.dispatch_visit calling visit_paragraph for Install it as a typical Python package: SphinxTranslator.dispatch_visit calling visit_Text for Install it as a typical Python package: SphinxTranslator.dispatch_departure calling depart_Text for Install it as a typical Python package: SphinxTranslator.dispatch_departure calling depart_paragraph for Install it as a typical Python package: SphinxTranslator.dispatch_visit calling visit_literal_block for $ python setup.py install SphinxTranslator.dispatch_departure calling depart_section for
InstallingInstall it as a typical Python package:$ python setup.py install
SphinxTranslator.dispatch_departure calling depart_section for
Installationpython-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested).Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too.This package supports Python 2.6, 2.7 and 3.3 or higher versions.
Installing from PyPI repositoryDo:$ pip install -U blosc
Building manuallyFirst, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources.Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not.
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
TestingAfter compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py:$ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode)Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite:$ nosetests --with-doctest (add -v for verbose mode)Once installed, you can re-run the tests at any time with:$ python -c "import blosc; blosc.test()"
InstallingInstall it as a typical Python package:$ python setup.py install
SphinxTranslator.dispatch_departure calling depart_document for
Installationpython-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested).Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too.This package supports Python 2.6, 2.7 and 3.3 or higher versions.
Installing from PyPI repositoryDo:$ pip install -U blosc
Building manuallyFirst, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources.Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not.
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
TestingAfter compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py:$ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode)Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite:$ nosetests --with-doctest (add -v for verbose mode)Once installed, you can re-run the tests at any time with:$ python -c "import blosc; blosc.test()"
InstallingInstall it as a typical Python package:$ python setup.py install
SphinxTranslator.dispatch_visit calling visit_document for Tutorials SphinxTranslator.dispatch_visit calling visit_title for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_title for Tutorials SphinxTranslator.dispatch_departure calling depart_document for Tutorials SphinxTranslator.dispatch_visit calling visit_document for Introduction SphinxTranslator.dispatch_visit calling visit_title for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_title for Introduction SphinxTranslator.dispatch_departure calling depart_document for Introduction SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for Installation SphinxTranslator.dispatch_visit calling visit_title for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_title for Installation SphinxTranslator.dispatch_departure calling depart_document for Installation SphinxTranslator.dispatch_visit calling visit_document for InstallationInstalling from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_visit calling visit_bullet_list for InstallationInstalling from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_visit calling visit_list_item for InstallationInstalling from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Installation SphinxTranslator.dispatch_visit calling visit_reference for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_reference for Installation SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Installation SphinxTranslator.dispatch_visit calling visit_bullet_list for Installing from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_visit calling visit_list_item for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_reference for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_Text for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_Text for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_reference for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_list_item for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_list_item for Building manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Building manually SphinxTranslator.dispatch_visit calling visit_reference for Building manually SphinxTranslator.dispatch_visit calling visit_Text for Building manually SphinxTranslator.dispatch_departure calling depart_Text for Building manually SphinxTranslator.dispatch_departure calling depart_reference for Building manually SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Building manually SphinxTranslator.dispatch_visit calling visit_bullet_list for Compiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_list_item for Compiling without an installed Blosc library SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Compiling without an installed Blosc library SphinxTranslator.dispatch_visit calling visit_reference for Compiling without an installed Blosc library SphinxTranslator.dispatch_visit calling visit_Text for Compiling without an installed Blosc library SphinxTranslator.dispatch_departure calling depart_Text for Compiling without an installed Blosc library SphinxTranslator.dispatch_departure calling depart_reference for Compiling without an installed Blosc library SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Compiling without an installed Blosc library SphinxTranslator.dispatch_departure calling depart_list_item for Compiling without an installed Blosc library SphinxTranslator.dispatch_visit calling visit_list_item for Compiling with an installed Blosc library SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Compiling with an installed Blosc library SphinxTranslator.dispatch_visit calling visit_reference for Compiling with an installed Blosc library SphinxTranslator.dispatch_visit calling visit_Text for Compiling with an installed Blosc library SphinxTranslator.dispatch_departure calling depart_Text for Compiling with an installed Blosc library SphinxTranslator.dispatch_departure calling depart_reference for Compiling with an installed Blosc library SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Compiling with an installed Blosc library SphinxTranslator.dispatch_departure calling depart_list_item for Compiling with an installed Blosc library SphinxTranslator.dispatch_visit calling visit_list_item for Generating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Generating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_reference for Generating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_Text for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_Text for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_reference for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_list_item for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_bullet_list for Compiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_list_item for Building manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_list_item for Testing SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Testing SphinxTranslator.dispatch_visit calling visit_reference for Testing SphinxTranslator.dispatch_visit calling visit_Text for Testing SphinxTranslator.dispatch_departure calling depart_Text for Testing SphinxTranslator.dispatch_departure calling depart_reference for Testing SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Testing SphinxTranslator.dispatch_departure calling depart_list_item for Testing SphinxTranslator.dispatch_visit calling visit_list_item for Installing SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Installing SphinxTranslator.dispatch_visit calling visit_reference for Installing SphinxTranslator.dispatch_visit calling visit_Text for Installing SphinxTranslator.dispatch_departure calling depart_Text for Installing SphinxTranslator.dispatch_departure calling depart_reference for Installing SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Installing SphinxTranslator.dispatch_departure calling depart_list_item for Installing SphinxTranslator.dispatch_departure calling depart_bullet_list for Installing from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_departure calling depart_list_item for InstallationInstalling from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_departure calling depart_bullet_list for InstallationInstalling from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_departure calling depart_document for InstallationInstalling from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling [app] emitting event: 'html-page-context'('install', 'page.html', {'parents': [], 'prev': {'link': '../intro/', 'title': 'Introduction'}, 'ne writing output... [ 60%] intro [app] emitting event: 'doctree-resolved'(>, 'intro') SphinxTranslator.dispatch_visit calling visit_document for Introduction SphinxTranslator.dispatch_visit calling visit_title for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_title for Introduction SphinxTranslator.dispatch_departure calling depart_document for Introduction SphinxTranslator.dispatch_visit calling visit_document for
Introduction
What is python-blosc?Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call.Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc.This is a Python package that wraps it.
Mailing listDiscussion about this module is welcome in the Blosc list:blosc@googlegroups.comhttp://groups.google.es/group/blosc
SphinxTranslator.dispatch_visit calling visit_section for
Introduction
What is python-blosc?Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call.Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc.This is a Python package that wraps it.
Mailing listDiscussion about this module is welcome in the Blosc list:blosc@googlegroups.comhttp://groups.google.es/group/blosc
SphinxTranslator.dispatch_visit calling visit_title for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_title for Introduction SphinxTranslator.dispatch_visit calling visit_section for
What is python-blosc?Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call.Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc.This is a Python package that wraps it.
SphinxTranslator.dispatch_visit calling visit_title for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_Text for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_Text for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_title for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_paragraph for Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. SphinxTranslator.dispatch_visit calling visit_Text for Blosc ( SphinxTranslator.dispatch_departure calling depart_Text for Blosc ( SphinxTranslator.dispatch_visit calling visit_reference for http://blosc.org SphinxTranslator.dispatch_visit calling visit_Text for http://blosc.org SphinxTranslator.dispatch_departure calling depart_Text for http://blosc.org SphinxTranslator.dispatch_departure calling depart_reference for http://blosc.org SphinxTranslator.dispatch_visit calling visit_Text for ) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. SphinxTranslator.dispatch_departure calling depart_Text for ) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. SphinxTranslator.dispatch_departure calling depart_paragraph for Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. SphinxTranslator.dispatch_visit calling visit_paragraph for Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. SphinxTranslator.dispatch_visit calling visit_Text for Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. SphinxTranslator.dispatch_departure calling depart_Text for Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. SphinxTranslator.dispatch_departure calling depart_paragraph for Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. SphinxTranslator.dispatch_visit calling visit_paragraph for This is a Python package that wraps it. SphinxTranslator.dispatch_visit calling visit_Text for This is a Python package that wraps it. SphinxTranslator.dispatch_departure calling depart_Text for This is a Python package that wraps it. SphinxTranslator.dispatch_departure calling depart_paragraph for This is a Python package that wraps it. SphinxTranslator.dispatch_departure calling depart_section for
What is python-blosc?Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call.Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc.This is a Python package that wraps it.
SphinxTranslator.dispatch_visit calling visit_section for
Mailing listDiscussion about this module is welcome in the Blosc list:blosc@googlegroups.comhttp://groups.google.es/group/blosc
SphinxTranslator.dispatch_visit calling visit_title for Mailing list SphinxTranslator.dispatch_visit calling visit_Text for Mailing list SphinxTranslator.dispatch_departure calling depart_Text for Mailing list SphinxTranslator.dispatch_departure calling depart_title for Mailing list SphinxTranslator.dispatch_visit calling visit_paragraph for Discussion about this module is welcome in the Blosc list: SphinxTranslator.dispatch_visit calling visit_Text for Discussion about this module is welcome in the Blosc list: SphinxTranslator.dispatch_departure calling depart_Text for Discussion about this module is welcome in the Blosc list: SphinxTranslator.dispatch_departure calling depart_paragraph for Discussion about this module is welcome in the Blosc list: SphinxTranslator.dispatch_visit calling visit_paragraph for blosc@googlegroups.com SphinxTranslator.dispatch_visit calling visit_reference for blosc@googlegroups.com SphinxTranslator.dispatch_visit calling visit_Text for blosc@googlegroups.com SphinxTranslator.dispatch_departure calling depart_Text for blosc@googlegroups.com SphinxTranslator.dispatch_departure calling depart_reference for blosc@googlegroups.com SphinxTranslator.dispatch_departure calling depart_paragraph for blosc@googlegroups.com SphinxTranslator.dispatch_visit calling visit_paragraph for http://groups.google.es/group/blosc SphinxTranslator.dispatch_visit calling visit_reference for http://groups.google.es/group/blosc SphinxTranslator.dispatch_visit calling visit_Text for http://groups.google.es/group/blosc SphinxTranslator.dispatch_departure calling depart_Text for http://groups.google.es/group/blosc SphinxTranslator.dispatch_departure calling depart_reference for http://groups.google.es/group/blosc SphinxTranslator.dispatch_departure calling depart_paragraph for http://groups.google.es/group/blosc SphinxTranslator.dispatch_departure calling depart_section for
Mailing listDiscussion about this module is welcome in the Blosc list:blosc@googlegroups.comhttp://groups.google.es/group/blosc
SphinxTranslator.dispatch_departure calling depart_section for
Introduction
What is python-blosc?Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call.Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc.This is a Python package that wraps it.
Mailing listDiscussion about this module is welcome in the Blosc list:blosc@googlegroups.comhttp://groups.google.es/group/blosc
SphinxTranslator.dispatch_departure calling depart_document for
Introduction
What is python-blosc?Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call.Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc.This is a Python package that wraps it.
Mailing listDiscussion about this module is welcome in the Blosc list:blosc@googlegroups.comhttp://groups.google.es/group/blosc
SphinxTranslator.dispatch_visit calling visit_document for Installation SphinxTranslator.dispatch_visit calling visit_title for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_title for Installation SphinxTranslator.dispatch_departure calling depart_document for Installation SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for Introduction SphinxTranslator.dispatch_visit calling visit_title for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_title for Introduction SphinxTranslator.dispatch_departure calling depart_document for Introduction SphinxTranslator.dispatch_visit calling visit_document for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_bullet_list for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_list_item for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Introduction SphinxTranslator.dispatch_visit calling visit_reference for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_reference for Introduction SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Introduction SphinxTranslator.dispatch_visit calling visit_bullet_list for What is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_list_item for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_compact_paragraph for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_reference for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_Text for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_Text for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_reference for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_compact_paragraph for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_list_item for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_list_item for Mailing list SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Mailing list SphinxTranslator.dispatch_visit calling visit_reference for Mailing list SphinxTranslator.dispatch_visit calling visit_Text for Mailing list SphinxTranslator.dispatch_departure calling depart_Text for Mailing list SphinxTranslator.dispatch_departure calling depart_reference for Mailing list SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Mailing list SphinxTranslator.dispatch_departure calling depart_list_item for Mailing list SphinxTranslator.dispatch_departure calling depart_bullet_list for What is python-blosc?Mailing list SphinxTranslator.dispatch_departure calling depart_list_item for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_departure calling depart_bullet_list for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_departure calling depart_document for IntroductionWhat is python-blosc?Mailing list [app] emitting event: 'html-page-context'('intro', 'page.html', {'parents': [], 'prev': {'link': '../', 'title': 'Welcome to python-blosc’s d writing output... [ 80%] reference [app] emitting event: 'doctree-resolved'(>, 'reference') SphinxTranslator.dispatch_visit calling visit_document for Library Reference SphinxTranslator.dispatch_visit calling visit_title for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_title for Library Reference SphinxTranslator.dispatch_departure calling depart_document for Library Reference SphinxTranslator.dispatch_visit calling visit_document for
Library Reference
First level variables__version__The version of the blosc package.blosclib_versionThe version of the Blosc C library.clib_versionsA map for the versions of the compression libraries included in C library.cnamesThe list of compressors included in C library.cname2clibA map between compressor names and its libraries (or formats).ncoresThe number of cores detected.
Public functionsblosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) Trueblosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans Trueblosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray Trueblosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize Trueblosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize Trueblosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True
Utilitiesblosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version.blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names.blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system.blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>>blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library.blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512)blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True)blosc.print_versionsPrint all the versions of software that python-blosc relies on.
SphinxTranslator.dispatch_visit calling visit_section for
Library Reference
First level variables__version__The version of the blosc package.blosclib_versionThe version of the Blosc C library.clib_versionsA map for the versions of the compression libraries included in C library.cnamesThe list of compressors included in C library.cname2clibA map between compressor names and its libraries (or formats).ncoresThe number of cores detected.
Public functionsblosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) Trueblosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans Trueblosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray Trueblosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize Trueblosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize Trueblosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True
Utilitiesblosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version.blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names.blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system.blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>>blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library.blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512)blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True)blosc.print_versionsPrint all the versions of software that python-blosc relies on.
SphinxTranslator.dispatch_visit calling visit_title for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_title for Library Reference SphinxTranslator.dispatch_visit calling visit_section for
First level variables__version__The version of the blosc package.blosclib_versionThe version of the Blosc C library.clib_versionsA map for the versions of the compression libraries included in C library.cnamesThe list of compressors included in C library.cname2clibA map between compressor names and its libraries (or formats).ncoresThe number of cores detected.
SphinxTranslator.dispatch_visit calling visit_title for First level variables SphinxTranslator.dispatch_visit calling visit_Text for First level variables SphinxTranslator.dispatch_departure calling depart_Text for First level variables SphinxTranslator.dispatch_departure calling depart_title for First level variables SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for __version__The version of the blosc package. SphinxTranslator.dispatch_visit calling visit_desc_signature for __version__ SphinxTranslator.dispatch_visit calling visit_desc_name for __version__ SphinxTranslator.dispatch_visit calling visit_Text for __version__ SphinxTranslator.dispatch_departure calling depart_Text for __version__ SphinxTranslator.dispatch_departure calling depart_desc_name for __version__ SphinxTranslator.dispatch_departure calling depart_desc_signature for __version__ SphinxTranslator.dispatch_visit calling visit_desc_content for The version of the blosc package. SphinxTranslator.dispatch_visit calling visit_paragraph for The version of the blosc package. SphinxTranslator.dispatch_visit calling visit_Text for The version of the blosc package. SphinxTranslator.dispatch_departure calling depart_Text for The version of the blosc package. SphinxTranslator.dispatch_departure calling depart_paragraph for The version of the blosc package. SphinxTranslator.dispatch_departure calling depart_desc_content for The version of the blosc package. SphinxTranslator.dispatch_departure calling depart_desc for __version__The version of the blosc package. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosclib_versionThe version of the Blosc C library. SphinxTranslator.dispatch_visit calling visit_desc_signature for blosclib_version SphinxTranslator.dispatch_visit calling visit_desc_name for blosclib_version SphinxTranslator.dispatch_visit calling visit_Text for blosclib_version SphinxTranslator.dispatch_departure calling depart_Text for blosclib_version SphinxTranslator.dispatch_departure calling depart_desc_name for blosclib_version SphinxTranslator.dispatch_departure calling depart_desc_signature for blosclib_version SphinxTranslator.dispatch_visit calling visit_desc_content for The version of the Blosc C library. SphinxTranslator.dispatch_visit calling visit_paragraph for The version of the Blosc C library. SphinxTranslator.dispatch_visit calling visit_Text for The version of the Blosc C library. SphinxTranslator.dispatch_departure calling depart_Text for The version of the Blosc C library. SphinxTranslator.dispatch_departure calling depart_paragraph for The version of the Blosc C library. SphinxTranslator.dispatch_departure calling depart_desc_content for The version of the Blosc C library. SphinxTranslator.dispatch_departure calling depart_desc for blosclib_versionThe version of the Blosc C library. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for clib_versionsA map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_visit calling visit_desc_signature for clib_versions SphinxTranslator.dispatch_visit calling visit_desc_name for clib_versions SphinxTranslator.dispatch_visit calling visit_Text for clib_versions SphinxTranslator.dispatch_departure calling depart_Text for clib_versions SphinxTranslator.dispatch_departure calling depart_desc_name for clib_versions SphinxTranslator.dispatch_departure calling depart_desc_signature for clib_versions SphinxTranslator.dispatch_visit calling visit_desc_content for A map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_visit calling visit_paragraph for A map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_visit calling visit_Text for A map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_departure calling depart_Text for A map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_departure calling depart_paragraph for A map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_departure calling depart_desc_content for A map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_departure calling depart_desc for clib_versionsA map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for cnamesThe list of compressors included in C library. SphinxTranslator.dispatch_visit calling visit_desc_signature for cnames SphinxTranslator.dispatch_visit calling visit_desc_name for cnames SphinxTranslator.dispatch_visit calling visit_Text for cnames SphinxTranslator.dispatch_departure calling depart_Text for cnames SphinxTranslator.dispatch_departure calling depart_desc_name for cnames SphinxTranslator.dispatch_departure calling depart_desc_signature for cnames SphinxTranslator.dispatch_visit calling visit_desc_content for The list of compressors included in C library. SphinxTranslator.dispatch_visit calling visit_paragraph for The list of compressors included in C library. SphinxTranslator.dispatch_visit calling visit_Text for The list of compressors included in C library. SphinxTranslator.dispatch_departure calling depart_Text for The list of compressors included in C library. SphinxTranslator.dispatch_departure calling depart_paragraph for The list of compressors included in C library. SphinxTranslator.dispatch_departure calling depart_desc_content for The list of compressors included in C library. SphinxTranslator.dispatch_departure calling depart_desc for cnamesThe list of compressors included in C library. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for cname2clibA map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_visit calling visit_desc_signature for cname2clib SphinxTranslator.dispatch_visit calling visit_desc_name for cname2clib SphinxTranslator.dispatch_visit calling visit_Text for cname2clib SphinxTranslator.dispatch_departure calling depart_Text for cname2clib SphinxTranslator.dispatch_departure calling depart_desc_name for cname2clib SphinxTranslator.dispatch_departure calling depart_desc_signature for cname2clib SphinxTranslator.dispatch_visit calling visit_desc_content for A map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_visit calling visit_paragraph for A map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_visit calling visit_Text for A map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_departure calling depart_Text for A map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_departure calling depart_paragraph for A map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_departure calling depart_desc_content for A map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_departure calling depart_desc for cname2clibA map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for ncoresThe number of cores detected. SphinxTranslator.dispatch_visit calling visit_desc_signature for ncores SphinxTranslator.dispatch_visit calling visit_desc_name for ncores SphinxTranslator.dispatch_visit calling visit_Text for ncores SphinxTranslator.dispatch_departure calling depart_Text for ncores SphinxTranslator.dispatch_departure calling depart_desc_name for ncores SphinxTranslator.dispatch_departure calling depart_desc_signature for ncores SphinxTranslator.dispatch_visit calling visit_desc_content for The number of cores detected. SphinxTranslator.dispatch_visit calling visit_paragraph for The number of cores detected. SphinxTranslator.dispatch_visit calling visit_Text for The number of cores detected. SphinxTranslator.dispatch_departure calling depart_Text for The number of cores detected. SphinxTranslator.dispatch_departure calling depart_paragraph for The number of cores detected. SphinxTranslator.dispatch_departure calling depart_desc_content for The number of cores detected. SphinxTranslator.dispatch_departure calling depart_desc for ncoresThe number of cores detected. SphinxTranslator.dispatch_departure calling depart_section for
First level variables__version__The version of the blosc package.blosclib_versionThe version of the Blosc C library.clib_versionsA map for the versions of the compression libraries included in C library.cnamesThe list of compressors included in C library.cname2clibA map between compressor names and its libraries (or formats).ncoresThe number of cores detected.
SphinxTranslator.dispatch_visit calling visit_section for
Public functionsblosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) Trueblosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans Trueblosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray Trueblosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize Trueblosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize Trueblosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True
SphinxTranslator.dispatch_visit calling visit_title for Public functions SphinxTranslator.dispatch_visit calling visit_Text for Public functions SphinxTranslator.dispatch_departure calling depart_Text for Public functions SphinxTranslator.dispatch_departure calling depart_title for Public functions SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) True SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for compress SphinxTranslator.dispatch_visit calling visit_Text for compress SphinxTranslator.dispatch_departure calling depart_Text for compress SphinxTranslator.dispatch_departure calling depart_desc_name for compress SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for bytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_parameter for bytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_Text for bytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_Text for bytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_parameter for bytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for bytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_content for Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) True SphinxTranslator.dispatch_visit calling visit_paragraph for Compress bytesobj, with a given type size. SphinxTranslator.dispatch_visit calling visit_Text for Compress bytesobj, with a given type size. SphinxTranslator.dispatch_departure calling depart_Text for Compress bytesobj, with a given type size. SphinxTranslator.dispatch_departure calling depart_paragraph for Compress bytesobj, with a given type size. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_visit calling visit_field for Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for bytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_paragraph for bytesobj : bytes-like object (supporting the buffer interface) SphinxTranslator.dispatch_visit calling visit_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for : bytes-like object (supporting the buffer interface) SphinxTranslator.dispatch_departure calling depart_Text for : bytes-like object (supporting the buffer interface) SphinxTranslator.dispatch_departure calling depart_paragraph for bytesobj : bytes-like object (supporting the buffer interface) SphinxTranslator.dispatch_visit calling visit_block_quote for The data to be compressed. SphinxTranslator.dispatch_visit calling visit_paragraph for The data to be compressed. SphinxTranslator.dispatch_visit calling visit_Text for The data to be compressed. SphinxTranslator.dispatch_departure calling depart_Text for The data to be compressed. SphinxTranslator.dispatch_departure calling depart_paragraph for The data to be compressed. SphinxTranslator.dispatch_departure calling depart_block_quote for The data to be compressed. SphinxTranslator.dispatch_visit calling visit_paragraph for typesize : int SphinxTranslator.dispatch_visit calling visit_strong for typesize SphinxTranslator.dispatch_visit calling visit_Text for typesize SphinxTranslator.dispatch_departure calling depart_Text for typesize SphinxTranslator.dispatch_departure calling depart_strong for typesize SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for typesize : int SphinxTranslator.dispatch_visit calling visit_block_quote for The data type size. SphinxTranslator.dispatch_visit calling visit_paragraph for The data type size. SphinxTranslator.dispatch_visit calling visit_Text for The data type size. SphinxTranslator.dispatch_departure calling depart_Text for The data type size. SphinxTranslator.dispatch_departure calling depart_paragraph for The data type size. SphinxTranslator.dispatch_departure calling depart_block_quote for The data type size. SphinxTranslator.dispatch_visit calling visit_paragraph for clevel : int (optional) SphinxTranslator.dispatch_visit calling visit_strong for clevel SphinxTranslator.dispatch_visit calling visit_Text for clevel SphinxTranslator.dispatch_departure calling depart_Text for clevel SphinxTranslator.dispatch_departure calling depart_strong for clevel SphinxTranslator.dispatch_visit calling visit_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for clevel : int (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_paragraph for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_Text for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_Text for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_paragraph for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_block_quote for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_paragraph for shuffle : int (optional) SphinxTranslator.dispatch_visit calling visit_strong for shuffle SphinxTranslator.dispatch_visit calling visit_Text for shuffle SphinxTranslator.dispatch_departure calling depart_Text for shuffle SphinxTranslator.dispatch_departure calling depart_strong for shuffle SphinxTranslator.dispatch_visit calling visit_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for shuffle : int (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_paragraph for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_Text for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_Text for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_paragraph for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_block_quote for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_paragraph for cname : string (optional) SphinxTranslator.dispatch_visit calling visit_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for cname SphinxTranslator.dispatch_departure calling depart_Text for cname SphinxTranslator.dispatch_departure calling depart_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for : string (optional) SphinxTranslator.dispatch_departure calling depart_Text for : string (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for cname : string (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_paragraph for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_Text for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_Text for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_paragraph for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_block_quote for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_field_body for bytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_field for Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for out : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for out : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_Text for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_Text for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_paragraph for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_block_quote for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_field_body for out : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_field for RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for TypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_visit calling visit_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_strong for TypeError SphinxTranslator.dispatch_visit calling visit_Text for TypeError SphinxTranslator.dispatch_departure calling depart_Text for TypeError SphinxTranslator.dispatch_departure calling depart_strong for TypeError SphinxTranslator.dispatch_departure calling depart_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_block_quote for If bytesobj doesn’t support the buffer interface. SphinxTranslator.dispatch_visit calling visit_paragraph for If bytesobj doesn’t support the buffer interface. SphinxTranslator.dispatch_visit calling visit_Text for If bytesobj doesn’t support the buffer interface. SphinxTranslator.dispatch_departure calling depart_Text for If bytesobj doesn’t support the buffer interface. SphinxTranslator.dispatch_departure calling depart_paragraph for If bytesobj doesn’t support the buffer interface. SphinxTranslator.dispatch_departure calling depart_block_quote for If bytesobj doesn’t support the buffer interface. SphinxTranslator.dispatch_visit calling visit_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_strong for ValueError SphinxTranslator.dispatch_visit calling visit_Text for ValueError SphinxTranslator.dispatch_departure calling depart_Text for ValueError SphinxTranslator.dispatch_departure calling depart_strong for ValueError SphinxTranslator.dispatch_departure calling depart_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_block_quote for If bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_visit calling visit_paragraph for If bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_visit calling visit_Text for If bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_departure calling depart_Text for If bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_departure calling depart_paragraph for If bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_departure calling depart_block_quote for If bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_departure calling depart_field_body for TypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_departure calling depart_field for RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) True SphinxTranslator.dispatch_departure calling depart_desc_content for Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) True SphinxTranslator.dispatch_departure calling depart_desc for blosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) True SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for compress_ptr SphinxTranslator.dispatch_visit calling visit_Text for compress_ptr SphinxTranslator.dispatch_departure calling depart_Text for compress_ptr SphinxTranslator.dispatch_departure calling depart_desc_name for compress_ptr SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for address, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_parameter for address, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_Text for address, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_Text for address, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_parameter for address, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for address, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_content for Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True SphinxTranslator.dispatch_visit calling visit_paragraph for Compress the data at address with given items and typesize. SphinxTranslator.dispatch_visit calling visit_Text for Compress the data at address with given items and typesize. SphinxTranslator.dispatch_departure calling depart_Text for Compress the data at address with given items and typesize. SphinxTranslator.dispatch_departure calling depart_paragraph for Compress the data at address with given items and typesize. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_field for Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for address : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_paragraph for address : int or long SphinxTranslator.dispatch_visit calling visit_strong for address SphinxTranslator.dispatch_visit calling visit_Text for address SphinxTranslator.dispatch_departure calling depart_Text for address SphinxTranslator.dispatch_departure calling depart_strong for address SphinxTranslator.dispatch_visit calling visit_Text for : int or long SphinxTranslator.dispatch_departure calling depart_Text for : int or long SphinxTranslator.dispatch_departure calling depart_paragraph for address : int or long SphinxTranslator.dispatch_visit calling visit_block_quote for the pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_paragraph for the pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_Text for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_Text for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_paragraph for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_block_quote for the pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_paragraph for items : int SphinxTranslator.dispatch_visit calling visit_strong for items SphinxTranslator.dispatch_visit calling visit_Text for items SphinxTranslator.dispatch_departure calling depart_Text for items SphinxTranslator.dispatch_departure calling depart_strong for items SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for items : int SphinxTranslator.dispatch_visit calling visit_block_quote for The number of items (of typesize) to be compressed. SphinxTranslator.dispatch_visit calling visit_paragraph for The number of items (of typesize) to be compressed. SphinxTranslator.dispatch_visit calling visit_Text for The number of items (of typesize) to be compressed. SphinxTranslator.dispatch_departure calling depart_Text for The number of items (of typesize) to be compressed. SphinxTranslator.dispatch_departure calling depart_paragraph for The number of items (of typesize) to be compressed. SphinxTranslator.dispatch_departure calling depart_block_quote for The number of items (of typesize) to be compressed. SphinxTranslator.dispatch_visit calling visit_paragraph for typesize : int SphinxTranslator.dispatch_visit calling visit_strong for typesize SphinxTranslator.dispatch_visit calling visit_Text for typesize SphinxTranslator.dispatch_departure calling depart_Text for typesize SphinxTranslator.dispatch_departure calling depart_strong for typesize SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for typesize : int SphinxTranslator.dispatch_visit calling visit_block_quote for The data type size. SphinxTranslator.dispatch_visit calling visit_paragraph for The data type size. SphinxTranslator.dispatch_visit calling visit_Text for The data type size. SphinxTranslator.dispatch_departure calling depart_Text for The data type size. SphinxTranslator.dispatch_departure calling depart_paragraph for The data type size. SphinxTranslator.dispatch_departure calling depart_block_quote for The data type size. SphinxTranslator.dispatch_visit calling visit_paragraph for clevel : int (optional) SphinxTranslator.dispatch_visit calling visit_strong for clevel SphinxTranslator.dispatch_visit calling visit_Text for clevel SphinxTranslator.dispatch_departure calling depart_Text for clevel SphinxTranslator.dispatch_departure calling depart_strong for clevel SphinxTranslator.dispatch_visit calling visit_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for clevel : int (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_paragraph for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_Text for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_Text for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_paragraph for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_block_quote for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_paragraph for shuffle : int (optional) SphinxTranslator.dispatch_visit calling visit_strong for shuffle SphinxTranslator.dispatch_visit calling visit_Text for shuffle SphinxTranslator.dispatch_departure calling depart_Text for shuffle SphinxTranslator.dispatch_departure calling depart_strong for shuffle SphinxTranslator.dispatch_visit calling visit_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for shuffle : int (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_paragraph for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_Text for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_Text for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_paragraph for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_block_quote for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_paragraph for cname : string (optional) SphinxTranslator.dispatch_visit calling visit_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for cname SphinxTranslator.dispatch_departure calling depart_Text for cname SphinxTranslator.dispatch_departure calling depart_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for : string (optional) SphinxTranslator.dispatch_departure calling depart_Text for : string (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for cname : string (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_paragraph for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_Text for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_Text for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_paragraph for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_block_quote for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_field_body for address : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_field for Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for out : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for out : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_Text for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_Text for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_paragraph for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_block_quote for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_field_body for out : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_field for RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for TypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_strong for TypeError SphinxTranslator.dispatch_visit calling visit_Text for TypeError SphinxTranslator.dispatch_departure calling depart_Text for TypeError SphinxTranslator.dispatch_departure calling depart_strong for TypeError SphinxTranslator.dispatch_departure calling depart_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_block_quote for If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_paragraph for If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_Text for If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_Text for If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_paragraph for If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_block_quote for If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_strong for ValueError SphinxTranslator.dispatch_visit calling visit_Text for ValueError SphinxTranslator.dispatch_departure calling depart_Text for ValueError SphinxTranslator.dispatch_departure calling depart_strong for ValueError SphinxTranslator.dispatch_departure calling depart_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_block_quote for If items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_paragraph for If items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_Text for If items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_Text for If items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_paragraph for If items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_block_quote for If items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_field_body for TypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_field for RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_rubric for Notes SphinxTranslator.dispatch_visit calling visit_Text for Notes SphinxTranslator.dispatch_departure calling depart_Text for Notes SphinxTranslator.dispatch_departure calling depart_rubric for Notes SphinxTranslator.dispatch_visit calling visit_paragraph for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_visit calling visit_Text for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_departure calling depart_Text for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_departure calling depart_paragraph for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_visit calling visit_paragraph for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_visit calling visit_Text for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_departure calling depart_Text for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_departure calling depart_paragraph for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True SphinxTranslator.dispatch_departure calling depart_desc_content for Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True SphinxTranslator.dispatch_departure calling depart_desc for blosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray True SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.decompressbytesobj SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for decompress SphinxTranslator.dispatch_visit calling visit_Text for decompress SphinxTranslator.dispatch_departure calling depart_Text for decompress SphinxTranslator.dispatch_departure calling depart_desc_name for decompress SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for bytesobj SphinxTranslator.dispatch_visit calling visit_desc_parameter for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_parameter for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.decompressbytesobj SphinxTranslator.dispatch_visit calling visit_desc_content for Decompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray True SphinxTranslator.dispatch_visit calling visit_paragraph for Decompresses a bytesobj compressed object. SphinxTranslator.dispatch_visit calling visit_Text for Decompresses a bytesobj compressed object. SphinxTranslator.dispatch_departure calling depart_Text for Decompresses a bytesobj compressed object. SphinxTranslator.dispatch_departure calling depart_paragraph for Decompresses a bytesobj compressed object. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_field for Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for bytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_visit calling visit_paragraph for bytesobj : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for bytesobj : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The data to be decompressed. SphinxTranslator.dispatch_visit calling visit_paragraph for The data to be decompressed. SphinxTranslator.dispatch_visit calling visit_Text for The data to be decompressed. SphinxTranslator.dispatch_departure calling depart_Text for The data to be decompressed. SphinxTranslator.dispatch_departure calling depart_paragraph for The data to be decompressed. SphinxTranslator.dispatch_departure calling depart_block_quote for The data to be decompressed. SphinxTranslator.dispatch_visit calling visit_paragraph for as_bytearray : bool, optional SphinxTranslator.dispatch_visit calling visit_strong for as_bytearray SphinxTranslator.dispatch_visit calling visit_Text for as_bytearray SphinxTranslator.dispatch_departure calling depart_Text for as_bytearray SphinxTranslator.dispatch_departure calling depart_strong for as_bytearray SphinxTranslator.dispatch_visit calling visit_Text for : bool, optional SphinxTranslator.dispatch_departure calling depart_Text for : bool, optional SphinxTranslator.dispatch_departure calling depart_paragraph for as_bytearray : bool, optional SphinxTranslator.dispatch_visit calling visit_block_quote for If this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_visit calling visit_paragraph for If this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_visit calling visit_Text for If this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_departure calling depart_Text for If this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_departure calling depart_paragraph for If this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_departure calling depart_block_quote for If this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_departure calling depart_field_body for bytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_departure calling depart_field for Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for out : str / bytes or bytearray SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes or bytearray SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes or bytearray SphinxTranslator.dispatch_departure calling depart_paragraph for out : str / bytes or bytearray SphinxTranslator.dispatch_visit calling visit_block_quote for The decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for The decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_visit calling visit_Text for The decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_departure calling depart_Text for The decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_departure calling depart_paragraph for The decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_departure calling depart_block_quote for The decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_departure calling depart_field_body for out : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_visit calling visit_field for RaisesTypeErrorIf bytesobj is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for TypeErrorIf bytesobj is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_strong for TypeError SphinxTranslator.dispatch_visit calling visit_Text for TypeError SphinxTranslator.dispatch_departure calling depart_Text for TypeError SphinxTranslator.dispatch_departure calling depart_strong for TypeError SphinxTranslator.dispatch_departure calling depart_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_block_quote for If bytesobj is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_paragraph for If bytesobj is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_Text for If bytesobj is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_Text for If bytesobj is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_paragraph for If bytesobj is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_block_quote for If bytesobj is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_field_body for TypeErrorIf bytesobj is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_field for RaisesTypeErrorIf bytesobj is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray True SphinxTranslator.dispatch_departure calling depart_desc_content for Decompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray True SphinxTranslator.dispatch_departure calling depart_desc for blosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray True SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize True SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.decompress_ptrbytesobjaddress SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for decompress_ptr SphinxTranslator.dispatch_visit calling visit_Text for decompress_ptr SphinxTranslator.dispatch_departure calling depart_Text for decompress_ptr SphinxTranslator.dispatch_departure calling depart_desc_name for decompress_ptr SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for bytesobjaddress SphinxTranslator.dispatch_visit calling visit_desc_parameter for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_parameter for bytesobj SphinxTranslator.dispatch_visit calling visit_desc_parameter for address SphinxTranslator.dispatch_visit calling visit_Text for address SphinxTranslator.dispatch_departure calling depart_Text for address SphinxTranslator.dispatch_departure calling depart_desc_parameter for address SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for bytesobjaddress SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.decompress_ptrbytesobjaddress SphinxTranslator.dispatch_visit calling visit_desc_content for Decompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize True SphinxTranslator.dispatch_visit calling visit_paragraph for Decompresses a bytesobj compressed object into the memory at address. SphinxTranslator.dispatch_visit calling visit_Text for Decompresses a bytesobj compressed object into the memory at address. SphinxTranslator.dispatch_departure calling depart_Text for Decompresses a bytesobj compressed object into the memory at address. SphinxTranslator.dispatch_departure calling depart_paragraph for Decompresses a bytesobj compressed object into the memory at address. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_field for Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for bytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_paragraph for bytesobj : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for bytesobj : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The data to be decompressed. SphinxTranslator.dispatch_visit calling visit_paragraph for The data to be decompressed. SphinxTranslator.dispatch_visit calling visit_Text for The data to be decompressed. SphinxTranslator.dispatch_departure calling depart_Text for The data to be decompressed. SphinxTranslator.dispatch_departure calling depart_paragraph for The data to be decompressed. SphinxTranslator.dispatch_departure calling depart_block_quote for The data to be decompressed. SphinxTranslator.dispatch_visit calling visit_paragraph for address : int or long SphinxTranslator.dispatch_visit calling visit_strong for address SphinxTranslator.dispatch_visit calling visit_Text for address SphinxTranslator.dispatch_departure calling depart_Text for address SphinxTranslator.dispatch_departure calling depart_strong for address SphinxTranslator.dispatch_visit calling visit_Text for : int or long SphinxTranslator.dispatch_departure calling depart_Text for : int or long SphinxTranslator.dispatch_departure calling depart_paragraph for address : int or long SphinxTranslator.dispatch_visit calling visit_block_quote for the pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_paragraph for the pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_Text for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_Text for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_paragraph for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_block_quote for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_field_body for bytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_field for Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_field for Returnsnbytes : intthe number of bytes written to the buffer SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for nbytes : intthe number of bytes written to the buffer SphinxTranslator.dispatch_visit calling visit_paragraph for nbytes : int SphinxTranslator.dispatch_visit calling visit_strong for nbytes SphinxTranslator.dispatch_visit calling visit_Text for nbytes SphinxTranslator.dispatch_departure calling depart_Text for nbytes SphinxTranslator.dispatch_departure calling depart_strong for nbytes SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for nbytes : int SphinxTranslator.dispatch_visit calling visit_block_quote for the number of bytes written to the buffer SphinxTranslator.dispatch_visit calling visit_paragraph for the number of bytes written to the buffer SphinxTranslator.dispatch_visit calling visit_Text for the number of bytes written to the buffer SphinxTranslator.dispatch_departure calling depart_Text for the number of bytes written to the buffer SphinxTranslator.dispatch_departure calling depart_paragraph for the number of bytes written to the buffer SphinxTranslator.dispatch_departure calling depart_block_quote for the number of bytes written to the buffer SphinxTranslator.dispatch_departure calling depart_field_body for nbytes : intthe number of bytes written to the buffer SphinxTranslator.dispatch_departure calling depart_field for Returnsnbytes : intthe number of bytes written to the buffer SphinxTranslator.dispatch_visit calling visit_field for RaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for TypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_strong for TypeError SphinxTranslator.dispatch_visit calling visit_Text for TypeError SphinxTranslator.dispatch_departure calling depart_Text for TypeError SphinxTranslator.dispatch_departure calling depart_strong for TypeError SphinxTranslator.dispatch_departure calling depart_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_block_quote for If bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_paragraph for If bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_Text for If bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_Text for If bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_paragraph for If bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_block_quote for If bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_field_body for TypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_field for RaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_rubric for Notes SphinxTranslator.dispatch_visit calling visit_Text for Notes SphinxTranslator.dispatch_departure calling depart_Text for Notes SphinxTranslator.dispatch_departure calling depart_rubric for Notes SphinxTranslator.dispatch_visit calling visit_paragraph for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_visit calling visit_Text for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_departure calling depart_Text for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_departure calling depart_paragraph for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_visit calling visit_paragraph for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_visit calling visit_Text for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_departure calling depart_Text for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_departure calling depart_paragraph for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize True SphinxTranslator.dispatch_departure calling depart_desc_content for Decompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize True SphinxTranslator.dispatch_departure calling depart_desc for blosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize True SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for pack_array SphinxTranslator.dispatch_visit calling visit_Text for pack_array SphinxTranslator.dispatch_departure calling depart_Text for pack_array SphinxTranslator.dispatch_departure calling depart_desc_name for pack_array SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for array[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_parameter for array[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_Text for array[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_Text for array[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_parameter for array[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for array[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_content for Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True SphinxTranslator.dispatch_visit calling visit_paragraph for Pack (compress) a NumPy array. SphinxTranslator.dispatch_visit calling visit_Text for Pack (compress) a NumPy array. SphinxTranslator.dispatch_departure calling depart_Text for Pack (compress) a NumPy array. SphinxTranslator.dispatch_departure calling depart_paragraph for Pack (compress) a NumPy array. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_field for Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for array : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_paragraph for array : ndarray SphinxTranslator.dispatch_visit calling visit_strong for array SphinxTranslator.dispatch_visit calling visit_Text for array SphinxTranslator.dispatch_departure calling depart_Text for array SphinxTranslator.dispatch_departure calling depart_strong for array SphinxTranslator.dispatch_visit calling visit_Text for : ndarray SphinxTranslator.dispatch_departure calling depart_Text for : ndarray SphinxTranslator.dispatch_departure calling depart_paragraph for array : ndarray SphinxTranslator.dispatch_visit calling visit_block_quote for The NumPy array to be packed. SphinxTranslator.dispatch_visit calling visit_paragraph for The NumPy array to be packed. SphinxTranslator.dispatch_visit calling visit_Text for The NumPy array to be packed. SphinxTranslator.dispatch_departure calling depart_Text for The NumPy array to be packed. SphinxTranslator.dispatch_departure calling depart_paragraph for The NumPy array to be packed. SphinxTranslator.dispatch_departure calling depart_block_quote for The NumPy array to be packed. SphinxTranslator.dispatch_visit calling visit_paragraph for clevel : int (optional) SphinxTranslator.dispatch_visit calling visit_strong for clevel SphinxTranslator.dispatch_visit calling visit_Text for clevel SphinxTranslator.dispatch_departure calling depart_Text for clevel SphinxTranslator.dispatch_departure calling depart_strong for clevel SphinxTranslator.dispatch_visit calling visit_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for clevel : int (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_paragraph for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_Text for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_Text for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_paragraph for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_block_quote for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_paragraph for shuffle : int (optional) SphinxTranslator.dispatch_visit calling visit_strong for shuffle SphinxTranslator.dispatch_visit calling visit_Text for shuffle SphinxTranslator.dispatch_departure calling depart_Text for shuffle SphinxTranslator.dispatch_departure calling depart_strong for shuffle SphinxTranslator.dispatch_visit calling visit_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for shuffle : int (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_paragraph for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_Text for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_Text for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_paragraph for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_block_quote for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_paragraph for cname : string (optional) SphinxTranslator.dispatch_visit calling visit_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for cname SphinxTranslator.dispatch_departure calling depart_Text for cname SphinxTranslator.dispatch_departure calling depart_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for : string (optional) SphinxTranslator.dispatch_departure calling depart_Text for : string (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for cname : string (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_paragraph for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_Text for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_Text for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_paragraph for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_block_quote for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_field_body for array : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_field for Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : str / bytesThe packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : str / bytesThe packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for out : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for out : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for The packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_Text for The packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_Text for The packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_paragraph for The packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_block_quote for The packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_field_body for out : str / bytesThe packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : str / bytesThe packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_field for RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for TypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_strong for TypeError SphinxTranslator.dispatch_visit calling visit_Text for TypeError SphinxTranslator.dispatch_departure calling depart_Text for TypeError SphinxTranslator.dispatch_departure calling depart_strong for TypeError SphinxTranslator.dispatch_departure calling depart_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_block_quote for If array does not quack like a numpy ndarray. SphinxTranslator.dispatch_visit calling visit_paragraph for If array does not quack like a numpy ndarray. SphinxTranslator.dispatch_visit calling visit_Text for If array does not quack like a numpy ndarray. SphinxTranslator.dispatch_departure calling depart_Text for If array does not quack like a numpy ndarray. SphinxTranslator.dispatch_departure calling depart_paragraph for If array does not quack like a numpy ndarray. SphinxTranslator.dispatch_departure calling depart_block_quote for If array does not quack like a numpy ndarray. SphinxTranslator.dispatch_visit calling visit_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_strong for ValueError SphinxTranslator.dispatch_visit calling visit_Text for ValueError SphinxTranslator.dispatch_departure calling depart_Text for ValueError SphinxTranslator.dispatch_departure calling depart_strong for ValueError SphinxTranslator.dispatch_departure calling depart_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_block_quote for If array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_definition_list for If array.itemsize * array.size is larger than the maximum allowedbuffer size. SphinxTranslator.dispatch_visit calling visit_definition_list_item for If array.itemsize * array.size is larger than the maximum allowedbuffer size. SphinxTranslator.dispatch_visit calling visit_term for If array.itemsize * array.size is larger than the maximum allowed SphinxTranslator.dispatch_visit calling visit_Text for If array.itemsize * array.size is larger than the maximum allowed SphinxTranslator.dispatch_departure calling depart_Text for If array.itemsize * array.size is larger than the maximum allowed SphinxTranslator.dispatch_departure calling depart_term for If array.itemsize * array.size is larger than the maximum allowed SphinxTranslator.dispatch_visit calling visit_definition for buffer size. SphinxTranslator.dispatch_visit calling visit_paragraph for buffer size. SphinxTranslator.dispatch_visit calling visit_Text for buffer size. SphinxTranslator.dispatch_departure calling depart_Text for buffer size. SphinxTranslator.dispatch_departure calling depart_paragraph for buffer size. SphinxTranslator.dispatch_departure calling depart_definition for buffer size. SphinxTranslator.dispatch_departure calling depart_definition_list_item for If array.itemsize * array.size is larger than the maximum allowedbuffer size. SphinxTranslator.dispatch_departure calling depart_definition_list for If array.itemsize * array.size is larger than the maximum allowedbuffer size. SphinxTranslator.dispatch_visit calling visit_paragraph for If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_Text for If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_Text for If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_paragraph for If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_block_quote for If array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_field_body for TypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_field for RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True SphinxTranslator.dispatch_departure calling depart_desc_content for Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True SphinxTranslator.dispatch_departure calling depart_desc for blosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.unpack_arraypacked_array SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for unpack_array SphinxTranslator.dispatch_visit calling visit_Text for unpack_array SphinxTranslator.dispatch_departure calling depart_Text for unpack_array SphinxTranslator.dispatch_departure calling depart_desc_name for unpack_array SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for packed_array SphinxTranslator.dispatch_visit calling visit_desc_parameter for packed_array SphinxTranslator.dispatch_visit calling visit_Text for packed_array SphinxTranslator.dispatch_departure calling depart_Text for packed_array SphinxTranslator.dispatch_departure calling depart_desc_parameter for packed_array SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for packed_array SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.unpack_arraypacked_array SphinxTranslator.dispatch_visit calling visit_desc_content for Unpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True SphinxTranslator.dispatch_visit calling visit_paragraph for Unpack (decompress) a packed NumPy array. SphinxTranslator.dispatch_visit calling visit_Text for Unpack (decompress) a packed NumPy array. SphinxTranslator.dispatch_departure calling depart_Text for Unpack (decompress) a packed NumPy array. SphinxTranslator.dispatch_departure calling depart_paragraph for Unpack (decompress) a packed NumPy array. SphinxTranslator.dispatch_visit calling visit_field_list for Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_field for Parameterspacked_array : str / bytesThe packed array to be decompressed. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for packed_array : str / bytesThe packed array to be decompressed. SphinxTranslator.dispatch_visit calling visit_paragraph for packed_array : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for packed_array SphinxTranslator.dispatch_visit calling visit_Text for packed_array SphinxTranslator.dispatch_departure calling depart_Text for packed_array SphinxTranslator.dispatch_departure calling depart_strong for packed_array SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for packed_array : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The packed array to be decompressed. SphinxTranslator.dispatch_visit calling visit_paragraph for The packed array to be decompressed. SphinxTranslator.dispatch_visit calling visit_Text for The packed array to be decompressed. SphinxTranslator.dispatch_departure calling depart_Text for The packed array to be decompressed. SphinxTranslator.dispatch_departure calling depart_paragraph for The packed array to be decompressed. SphinxTranslator.dispatch_departure calling depart_block_quote for The packed array to be decompressed. SphinxTranslator.dispatch_departure calling depart_field_body for packed_array : str / bytesThe packed array to be decompressed. SphinxTranslator.dispatch_departure calling depart_field for Parameterspacked_array : str / bytesThe packed array to be decompressed. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : ndarrayThe decompressed data in form of a NumPy array. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : ndarrayThe decompressed data in form of a NumPy array. SphinxTranslator.dispatch_visit calling visit_paragraph for out : ndarray SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : ndarray SphinxTranslator.dispatch_departure calling depart_Text for : ndarray SphinxTranslator.dispatch_departure calling depart_paragraph for out : ndarray SphinxTranslator.dispatch_visit calling visit_block_quote for The decompressed data in form of a NumPy array. SphinxTranslator.dispatch_visit calling visit_paragraph for The decompressed data in form of a NumPy array. SphinxTranslator.dispatch_visit calling visit_Text for The decompressed data in form of a NumPy array. SphinxTranslator.dispatch_departure calling depart_Text for The decompressed data in form of a NumPy array. SphinxTranslator.dispatch_departure calling depart_paragraph for The decompressed data in form of a NumPy array. SphinxTranslator.dispatch_departure calling depart_block_quote for The decompressed data in form of a NumPy array. SphinxTranslator.dispatch_departure calling depart_field_body for out : ndarrayThe decompressed data in form of a NumPy array. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : ndarrayThe decompressed data in form of a NumPy array. SphinxTranslator.dispatch_visit calling visit_field for RaisesTypeErrorIf packed_array is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for TypeErrorIf packed_array is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_strong for TypeError SphinxTranslator.dispatch_visit calling visit_Text for TypeError SphinxTranslator.dispatch_departure calling depart_Text for TypeError SphinxTranslator.dispatch_departure calling depart_strong for TypeError SphinxTranslator.dispatch_departure calling depart_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_block_quote for If packed_array is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_paragraph for If packed_array is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_Text for If packed_array is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_Text for If packed_array is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_paragraph for If packed_array is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_block_quote for If packed_array is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_field_body for TypeErrorIf packed_array is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_field for RaisesTypeErrorIf packed_array is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_field_list for Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True SphinxTranslator.dispatch_departure calling depart_desc_content for Unpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True SphinxTranslator.dispatch_departure calling depart_desc for blosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True SphinxTranslator.dispatch_departure calling depart_section for
Public functionsblosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) Trueblosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans Trueblosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray Trueblosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize Trueblosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize Trueblosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True
SphinxTranslator.dispatch_visit calling visit_section for
Utilitiesblosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version.blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names.blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system.blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>>blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library.blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512)blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True)blosc.print_versionsPrint all the versions of software that python-blosc relies on.
SphinxTranslator.dispatch_visit calling visit_title for Utilities SphinxTranslator.dispatch_visit calling visit_Text for Utilities SphinxTranslator.dispatch_departure calling depart_Text for Utilities SphinxTranslator.dispatch_departure calling depart_title for Utilities SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.clib_infocname SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for clib_info SphinxTranslator.dispatch_visit calling visit_Text for clib_info SphinxTranslator.dispatch_departure calling depart_Text for clib_info SphinxTranslator.dispatch_departure calling depart_desc_name for clib_info SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for cname SphinxTranslator.dispatch_visit calling visit_desc_parameter for cname SphinxTranslator.dispatch_visit calling visit_Text for cname SphinxTranslator.dispatch_departure calling depart_Text for cname SphinxTranslator.dispatch_departure calling depart_desc_parameter for cname SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for cname SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.clib_infocname SphinxTranslator.dispatch_visit calling visit_desc_content for Return info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_visit calling visit_paragraph for Return info for compression libraries in C library. SphinxTranslator.dispatch_visit calling visit_Text for Return info for compression libraries in C library. SphinxTranslator.dispatch_departure calling depart_Text for Return info for compression libraries in C library. SphinxTranslator.dispatch_departure calling depart_paragraph for Return info for compression libraries in C library. SphinxTranslator.dispatch_visit calling visit_field_list for Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_visit calling visit_field for Parameterscname : strThe compressor name. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for cname : strThe compressor name. SphinxTranslator.dispatch_visit calling visit_paragraph for cname : str SphinxTranslator.dispatch_visit calling visit_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for cname SphinxTranslator.dispatch_departure calling depart_Text for cname SphinxTranslator.dispatch_departure calling depart_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for : str SphinxTranslator.dispatch_departure calling depart_Text for : str SphinxTranslator.dispatch_departure calling depart_paragraph for cname : str SphinxTranslator.dispatch_visit calling visit_block_quote for The compressor name. SphinxTranslator.dispatch_visit calling visit_paragraph for The compressor name. SphinxTranslator.dispatch_visit calling visit_Text for The compressor name. SphinxTranslator.dispatch_departure calling depart_Text for The compressor name. SphinxTranslator.dispatch_departure calling depart_paragraph for The compressor name. SphinxTranslator.dispatch_departure calling depart_block_quote for The compressor name. SphinxTranslator.dispatch_departure calling depart_field_body for cname : strThe compressor name. SphinxTranslator.dispatch_departure calling depart_field for Parameterscname : strThe compressor name. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : tupleThe associated library name and version. SphinxTranslator.dispatch_visit calling visit_paragraph for out : tuple SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : tuple SphinxTranslator.dispatch_departure calling depart_Text for : tuple SphinxTranslator.dispatch_departure calling depart_paragraph for out : tuple SphinxTranslator.dispatch_visit calling visit_block_quote for The associated library name and version. SphinxTranslator.dispatch_visit calling visit_paragraph for The associated library name and version. SphinxTranslator.dispatch_visit calling visit_Text for The associated library name and version. SphinxTranslator.dispatch_departure calling depart_Text for The associated library name and version. SphinxTranslator.dispatch_departure calling depart_paragraph for The associated library name and version. SphinxTranslator.dispatch_departure calling depart_block_quote for The associated library name and version. SphinxTranslator.dispatch_departure calling depart_field_body for out : tupleThe associated library name and version. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_departure calling depart_field_list for Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_departure calling depart_desc_content for Return info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_departure calling depart_desc for blosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names. SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.compressor_list SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for compressor_list SphinxTranslator.dispatch_visit calling visit_Text for compressor_list SphinxTranslator.dispatch_departure calling depart_Text for compressor_list SphinxTranslator.dispatch_departure calling depart_desc_name for compressor_list SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.compressor_list SphinxTranslator.dispatch_visit calling visit_desc_content for Returns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names. SphinxTranslator.dispatch_visit calling visit_paragraph for Returns a list of compressors available in C library. SphinxTranslator.dispatch_visit calling visit_Text for Returns a list of compressors available in C library. SphinxTranslator.dispatch_departure calling depart_Text for Returns a list of compressors available in C library. SphinxTranslator.dispatch_departure calling depart_paragraph for Returns a list of compressors available in C library. SphinxTranslator.dispatch_visit calling visit_field_list for ParametersNoneReturnsout : listThe list of names. SphinxTranslator.dispatch_visit calling visit_field for ParametersNone SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for None SphinxTranslator.dispatch_visit calling visit_paragraph for None SphinxTranslator.dispatch_visit calling visit_strong for None SphinxTranslator.dispatch_visit calling visit_Text for None SphinxTranslator.dispatch_departure calling depart_Text for None SphinxTranslator.dispatch_departure calling depart_strong for None SphinxTranslator.dispatch_departure calling depart_paragraph for None SphinxTranslator.dispatch_departure calling depart_field_body for None SphinxTranslator.dispatch_departure calling depart_field for ParametersNone SphinxTranslator.dispatch_visit calling visit_field for Returnsout : listThe list of names. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : listThe list of names. SphinxTranslator.dispatch_visit calling visit_paragraph for out : list SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : list SphinxTranslator.dispatch_departure calling depart_Text for : list SphinxTranslator.dispatch_departure calling depart_paragraph for out : list SphinxTranslator.dispatch_visit calling visit_block_quote for The list of names. SphinxTranslator.dispatch_visit calling visit_paragraph for The list of names. SphinxTranslator.dispatch_visit calling visit_Text for The list of names. SphinxTranslator.dispatch_departure calling depart_Text for The list of names. SphinxTranslator.dispatch_departure calling depart_paragraph for The list of names. SphinxTranslator.dispatch_departure calling depart_block_quote for The list of names. SphinxTranslator.dispatch_departure calling depart_field_body for out : listThe list of names. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : listThe list of names. SphinxTranslator.dispatch_departure calling depart_field_list for ParametersNoneReturnsout : listThe list of names. SphinxTranslator.dispatch_departure calling depart_desc_content for Returns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names. SphinxTranslator.dispatch_departure calling depart_desc for blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.detect_number_of_cores SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for detect_number_of_cores SphinxTranslator.dispatch_visit calling visit_Text for detect_number_of_cores SphinxTranslator.dispatch_departure calling depart_Text for detect_number_of_cores SphinxTranslator.dispatch_departure calling depart_desc_name for detect_number_of_cores SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.detect_number_of_cores SphinxTranslator.dispatch_visit calling visit_desc_content for Detect the number of cores in this system.Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_visit calling visit_paragraph for Detect the number of cores in this system. SphinxTranslator.dispatch_visit calling visit_Text for Detect the number of cores in this system. SphinxTranslator.dispatch_departure calling depart_Text for Detect the number of cores in this system. SphinxTranslator.dispatch_departure calling depart_paragraph for Detect the number of cores in this system. SphinxTranslator.dispatch_visit calling visit_field_list for Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : intThe number of cores in this system. SphinxTranslator.dispatch_visit calling visit_paragraph for out : int SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for out : int SphinxTranslator.dispatch_visit calling visit_block_quote for The number of cores in this system. SphinxTranslator.dispatch_visit calling visit_paragraph for The number of cores in this system. SphinxTranslator.dispatch_visit calling visit_Text for The number of cores in this system. SphinxTranslator.dispatch_departure calling depart_Text for The number of cores in this system. SphinxTranslator.dispatch_departure calling depart_paragraph for The number of cores in this system. SphinxTranslator.dispatch_departure calling depart_block_quote for The number of cores in this system. SphinxTranslator.dispatch_departure calling depart_field_body for out : intThe number of cores in this system. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_departure calling depart_field_list for Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_departure calling depart_desc_content for Detect the number of cores in this system.Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_departure calling depart_desc for blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>> SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.free_resources SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for free_resources SphinxTranslator.dispatch_visit calling visit_Text for free_resources SphinxTranslator.dispatch_departure calling depart_Text for free_resources SphinxTranslator.dispatch_departure calling depart_desc_name for free_resources SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.free_resources SphinxTranslator.dispatch_visit calling visit_desc_content for Free possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>> SphinxTranslator.dispatch_visit calling visit_paragraph for Free possible memory temporaries and thread resources. SphinxTranslator.dispatch_visit calling visit_Text for Free possible memory temporaries and thread resources. SphinxTranslator.dispatch_departure calling depart_Text for Free possible memory temporaries and thread resources. SphinxTranslator.dispatch_departure calling depart_paragraph for Free possible memory temporaries and thread resources. SphinxTranslator.dispatch_visit calling visit_field_list for Returnsout : None SphinxTranslator.dispatch_visit calling visit_field for Returnsout : None SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : None SphinxTranslator.dispatch_visit calling visit_paragraph for out : None SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : None SphinxTranslator.dispatch_departure calling depart_Text for : None SphinxTranslator.dispatch_departure calling depart_paragraph for out : None SphinxTranslator.dispatch_departure calling depart_field_body for out : None SphinxTranslator.dispatch_departure calling depart_field for Returnsout : None SphinxTranslator.dispatch_departure calling depart_field_list for Returnsout : None SphinxTranslator.dispatch_visit calling visit_rubric for Notes SphinxTranslator.dispatch_visit calling visit_Text for Notes SphinxTranslator.dispatch_departure calling depart_Text for Notes SphinxTranslator.dispatch_departure calling depart_rubric for Notes SphinxTranslator.dispatch_visit calling visit_paragraph for Blosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while. SphinxTranslator.dispatch_visit calling visit_Text for Blosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while. SphinxTranslator.dispatch_departure calling depart_Text for Blosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while. SphinxTranslator.dispatch_departure calling depart_paragraph for Blosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> blosc.free_resources() >>> SphinxTranslator.dispatch_departure calling depart_desc_content for Free possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>> SphinxTranslator.dispatch_departure calling depart_desc for blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>> SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.get_clibbytesobj SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for get_clib SphinxTranslator.dispatch_visit calling visit_Text for get_clib SphinxTranslator.dispatch_departure calling depart_Text for get_clib SphinxTranslator.dispatch_departure calling depart_desc_name for get_clib SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for bytesobj SphinxTranslator.dispatch_visit calling visit_desc_parameter for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_parameter for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.get_clibbytesobj SphinxTranslator.dispatch_visit calling visit_desc_content for Return the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_visit calling visit_paragraph for Return the name of the compression library for Blosc bytesobj buffer. SphinxTranslator.dispatch_visit calling visit_Text for Return the name of the compression library for Blosc SphinxTranslator.dispatch_departure calling depart_Text for Return the name of the compression library for Blosc SphinxTranslator.dispatch_visit calling visit_literal for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_literal for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for buffer. SphinxTranslator.dispatch_departure calling depart_Text for buffer. SphinxTranslator.dispatch_departure calling depart_paragraph for Return the name of the compression library for Blosc bytesobj buffer. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_visit calling visit_field for Parametersbytesobj : str / bytesThe compressed buffer. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for bytesobj : str / bytesThe compressed buffer. SphinxTranslator.dispatch_visit calling visit_paragraph for bytesobj : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for bytesobj : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The compressed buffer. SphinxTranslator.dispatch_visit calling visit_paragraph for The compressed buffer. SphinxTranslator.dispatch_visit calling visit_Text for The compressed buffer. SphinxTranslator.dispatch_departure calling depart_Text for The compressed buffer. SphinxTranslator.dispatch_departure calling depart_paragraph for The compressed buffer. SphinxTranslator.dispatch_departure calling depart_block_quote for The compressed buffer. SphinxTranslator.dispatch_departure calling depart_field_body for bytesobj : str / bytesThe compressed buffer. SphinxTranslator.dispatch_departure calling depart_field for Parametersbytesobj : str / bytesThe compressed buffer. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : strThe name of the compression library. SphinxTranslator.dispatch_visit calling visit_paragraph for out : str SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : str SphinxTranslator.dispatch_departure calling depart_Text for : str SphinxTranslator.dispatch_departure calling depart_paragraph for out : str SphinxTranslator.dispatch_visit calling visit_block_quote for The name of the compression library. SphinxTranslator.dispatch_visit calling visit_paragraph for The name of the compression library. SphinxTranslator.dispatch_visit calling visit_Text for The name of the compression library. SphinxTranslator.dispatch_departure calling depart_Text for The name of the compression library. SphinxTranslator.dispatch_departure calling depart_paragraph for The name of the compression library. SphinxTranslator.dispatch_departure calling depart_block_quote for The name of the compression library. SphinxTranslator.dispatch_departure calling depart_field_body for out : strThe name of the compression library. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_departure calling depart_desc_content for Return the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_departure calling depart_desc for blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512) SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.set_blocksizeblocksize SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for set_blocksize SphinxTranslator.dispatch_visit calling visit_Text for set_blocksize SphinxTranslator.dispatch_departure calling depart_Text for set_blocksize SphinxTranslator.dispatch_departure calling depart_desc_name for set_blocksize SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for blocksize SphinxTranslator.dispatch_visit calling visit_desc_parameter for blocksize SphinxTranslator.dispatch_visit calling visit_Text for blocksize SphinxTranslator.dispatch_departure calling depart_Text for blocksize SphinxTranslator.dispatch_departure calling depart_desc_parameter for blocksize SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for blocksize SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.set_blocksizeblocksize SphinxTranslator.dispatch_visit calling visit_desc_content for Force the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512) SphinxTranslator.dispatch_visit calling visit_paragraph for Force the use of a specific blocksize. If 0, an automatic blocksize will be used (the default). SphinxTranslator.dispatch_visit calling visit_Text for Force the use of a specific blocksize. If 0, an automatic blocksize will be used (the default). SphinxTranslator.dispatch_departure calling depart_Text for Force the use of a specific blocksize. If 0, an automatic blocksize will be used (the default). SphinxTranslator.dispatch_departure calling depart_paragraph for Force the use of a specific blocksize. If 0, an automatic blocksize will be used (the default). SphinxTranslator.dispatch_visit calling visit_rubric for Notes SphinxTranslator.dispatch_visit calling visit_Text for Notes SphinxTranslator.dispatch_departure calling depart_Text for Notes SphinxTranslator.dispatch_departure calling depart_rubric for Notes SphinxTranslator.dispatch_visit calling visit_paragraph for This is a low-level function and is recommened for expert users only. SphinxTranslator.dispatch_visit calling visit_Text for This is a low-level function and is recommened for expert users only. SphinxTranslator.dispatch_departure calling depart_Text for This is a low-level function and is recommened for expert users only. SphinxTranslator.dispatch_departure calling depart_paragraph for This is a low-level function and is recommened for expert users only. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> blosc.set_blocksize(512) SphinxTranslator.dispatch_departure calling depart_desc_content for Force the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512) SphinxTranslator.dispatch_departure calling depart_desc for blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512) SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2 SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.set_nthreadsnthreads SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for set_nthreads SphinxTranslator.dispatch_visit calling visit_Text for set_nthreads SphinxTranslator.dispatch_departure calling depart_Text for set_nthreads SphinxTranslator.dispatch_departure calling depart_desc_name for set_nthreads SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for nthreads SphinxTranslator.dispatch_visit calling visit_desc_parameter for nthreads SphinxTranslator.dispatch_visit calling visit_Text for nthreads SphinxTranslator.dispatch_departure calling depart_Text for nthreads SphinxTranslator.dispatch_departure calling depart_desc_parameter for nthreads SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for nthreads SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.set_nthreadsnthreads SphinxTranslator.dispatch_visit calling visit_desc_content for Set the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2 SphinxTranslator.dispatch_visit calling visit_paragraph for Set the number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_Text for Set the number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_Text for Set the number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_paragraph for Set the number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_visit calling visit_field for Parametersnthreads : intThe number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for nthreads : intThe number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_paragraph for nthreads : int SphinxTranslator.dispatch_visit calling visit_strong for nthreads SphinxTranslator.dispatch_visit calling visit_Text for nthreads SphinxTranslator.dispatch_departure calling depart_Text for nthreads SphinxTranslator.dispatch_departure calling depart_strong for nthreads SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for nthreads : int SphinxTranslator.dispatch_visit calling visit_block_quote for The number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_paragraph for The number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_Text for The number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_Text for The number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_paragraph for The number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_block_quote for The number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_field_body for nthreads : intThe number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_field for Parametersnthreads : intThe number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : intThe previous number of used threads. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : intThe previous number of used threads. SphinxTranslator.dispatch_visit calling visit_paragraph for out : int SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for out : int SphinxTranslator.dispatch_visit calling visit_block_quote for The previous number of used threads. SphinxTranslator.dispatch_visit calling visit_paragraph for The previous number of used threads. SphinxTranslator.dispatch_visit calling visit_Text for The previous number of used threads. SphinxTranslator.dispatch_departure calling depart_Text for The previous number of used threads. SphinxTranslator.dispatch_departure calling depart_paragraph for The previous number of used threads. SphinxTranslator.dispatch_departure calling depart_block_quote for The previous number of used threads. SphinxTranslator.dispatch_departure calling depart_field_body for out : intThe previous number of used threads. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : intThe previous number of used threads. SphinxTranslator.dispatch_visit calling visit_field for RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for ValueErrorIf nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_visit calling visit_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_strong for ValueError SphinxTranslator.dispatch_visit calling visit_Text for ValueError SphinxTranslator.dispatch_departure calling depart_Text for ValueError SphinxTranslator.dispatch_departure calling depart_strong for ValueError SphinxTranslator.dispatch_departure calling depart_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_block_quote for If nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_visit calling visit_paragraph for If nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_visit calling visit_Text for If nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_departure calling depart_Text for If nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_departure calling depart_paragraph for If nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_departure calling depart_block_quote for If nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_departure calling depart_field_body for ValueErrorIf nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_departure calling depart_field for RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_visit calling visit_rubric for Notes SphinxTranslator.dispatch_visit calling visit_Text for Notes SphinxTranslator.dispatch_departure calling depart_Text for Notes SphinxTranslator.dispatch_departure calling depart_rubric for Notes SphinxTranslator.dispatch_visit calling visit_paragraph for The number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores. SphinxTranslator.dispatch_visit calling visit_Text for The number of threads for Blosc is the maximum number of cores detected on your machine (via SphinxTranslator.dispatch_departure calling depart_Text for The number of threads for Blosc is the maximum number of cores detected on your machine (via SphinxTranslator.dispatch_visit calling visit_literal for detect_number_of_cores SphinxTranslator.dispatch_visit calling visit_Text for detect_number_of_cores SphinxTranslator.dispatch_departure calling depart_Text for detect_number_of_cores SphinxTranslator.dispatch_departure calling depart_literal for detect_number_of_cores SphinxTranslator.dispatch_visit calling visit_Text for ). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores. SphinxTranslator.dispatch_departure calling depart_Text for ). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores. SphinxTranslator.dispatch_departure calling depart_paragraph for The number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_paragraph for Set the number of threads to 2 and then to 1: SphinxTranslator.dispatch_visit calling visit_Text for Set the number of threads to 2 and then to 1: SphinxTranslator.dispatch_departure calling depart_Text for Set the number of threads to 2 and then to 1: SphinxTranslator.dispatch_departure calling depart_paragraph for Set the number of threads to 2 and then to 1: SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2 SphinxTranslator.dispatch_departure calling depart_desc_content for Set the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2 SphinxTranslator.dispatch_departure calling depart_desc for blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2 SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True) SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.set_releasegilgitstate SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for set_releasegil SphinxTranslator.dispatch_visit calling visit_Text for set_releasegil SphinxTranslator.dispatch_departure calling depart_Text for set_releasegil SphinxTranslator.dispatch_departure calling depart_desc_name for set_releasegil SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for gitstate SphinxTranslator.dispatch_visit calling visit_desc_parameter for gitstate SphinxTranslator.dispatch_visit calling visit_Text for gitstate SphinxTranslator.dispatch_departure calling depart_Text for gitstate SphinxTranslator.dispatch_departure calling depart_desc_parameter for gitstate SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for gitstate SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.set_releasegilgitstate SphinxTranslator.dispatch_visit calling visit_desc_content for Sets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True) SphinxTranslator.dispatch_visit calling visit_paragraph for Sets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False. SphinxTranslator.dispatch_visit calling visit_Text for Sets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False. SphinxTranslator.dispatch_departure calling depart_Text for Sets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False. SphinxTranslator.dispatch_departure calling depart_paragraph for Sets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False. SphinxTranslator.dispatch_visit calling visit_rubric for Notes SphinxTranslator.dispatch_visit calling visit_Text for Notes SphinxTranslator.dispatch_departure calling depart_Text for Notes SphinxTranslator.dispatch_departure calling depart_rubric for Notes SphinxTranslator.dispatch_visit calling visit_paragraph for Designed to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes. SphinxTranslator.dispatch_visit calling visit_Text for Designed to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes. SphinxTranslator.dispatch_departure calling depart_Text for Designed to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes. SphinxTranslator.dispatch_departure calling depart_paragraph for Designed to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> oldReleaseState = blosc.set_releasegil(True) SphinxTranslator.dispatch_departure calling depart_desc_content for Sets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True) SphinxTranslator.dispatch_departure calling depart_desc for blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True) SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.print_versionsPrint all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.print_versions SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for print_versions SphinxTranslator.dispatch_visit calling visit_Text for print_versions SphinxTranslator.dispatch_departure calling depart_Text for print_versions SphinxTranslator.dispatch_departure calling depart_desc_name for print_versions SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.print_versions SphinxTranslator.dispatch_visit calling visit_desc_content for Print all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_visit calling visit_paragraph for Print all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_visit calling visit_Text for Print all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_departure calling depart_Text for Print all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_departure calling depart_paragraph for Print all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_departure calling depart_desc_content for Print all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_departure calling depart_desc for blosc.print_versionsPrint all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_departure calling depart_section for
Utilitiesblosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version.blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names.blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system.blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>>blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library.blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512)blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True)blosc.print_versionsPrint all the versions of software that python-blosc relies on.
SphinxTranslator.dispatch_departure calling depart_section for
Library Reference
First level variables__version__The version of the blosc package.blosclib_versionThe version of the Blosc C library.clib_versionsA map for the versions of the compression libraries included in C library.cnamesThe list of compressors included in C library.cname2clibA map between compressor names and its libraries (or formats).ncoresThe number of cores detected.
Public functionsblosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) Trueblosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans Trueblosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray Trueblosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize Trueblosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize Trueblosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True
Utilitiesblosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version.blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names.blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system.blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>>blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library.blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512)blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True)blosc.print_versionsPrint all the versions of software that python-blosc relies on.
SphinxTranslator.dispatch_departure calling depart_document for
Library Reference
First level variables__version__The version of the blosc package.blosclib_versionThe version of the Blosc C library.clib_versionsA map for the versions of the compression libraries included in C library.cnamesThe list of compressors included in C library.cname2clibA map between compressor names and its libraries (or formats).ncoresThe number of cores detected.
Public functionsblosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) Trueblosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans Trueblosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray Trueblosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize Trueblosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize Trueblosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True
Utilitiesblosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version.blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names.blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system.blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>>blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library.blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512)blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True)blosc.print_versionsPrint all the versions of software that python-blosc relies on.
SphinxTranslator.dispatch_visit calling visit_document for Tutorials SphinxTranslator.dispatch_visit calling visit_title for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_title for Tutorials SphinxTranslator.dispatch_departure calling depart_document for Tutorials SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for Library Reference SphinxTranslator.dispatch_visit calling visit_title for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_title for Library Reference SphinxTranslator.dispatch_departure calling depart_document for Library Reference SphinxTranslator.dispatch_visit calling visit_document for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_bullet_list for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_list_item for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Library Reference SphinxTranslator.dispatch_visit calling visit_reference for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_reference for Library Reference SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Library Reference SphinxTranslator.dispatch_visit calling visit_bullet_list for First level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_list_item for First level variables SphinxTranslator.dispatch_visit calling visit_compact_paragraph for First level variables SphinxTranslator.dispatch_visit calling visit_reference for First level variables SphinxTranslator.dispatch_visit calling visit_Text for First level variables SphinxTranslator.dispatch_departure calling depart_Text for First level variables SphinxTranslator.dispatch_departure calling depart_reference for First level variables SphinxTranslator.dispatch_departure calling depart_compact_paragraph for First level variables SphinxTranslator.dispatch_departure calling depart_list_item for First level variables SphinxTranslator.dispatch_visit calling visit_list_item for Public functions SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Public functions SphinxTranslator.dispatch_visit calling visit_reference for Public functions SphinxTranslator.dispatch_visit calling visit_Text for Public functions SphinxTranslator.dispatch_departure calling depart_Text for Public functions SphinxTranslator.dispatch_departure calling depart_reference for Public functions SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Public functions SphinxTranslator.dispatch_departure calling depart_list_item for Public functions SphinxTranslator.dispatch_visit calling visit_list_item for Utilities SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Utilities SphinxTranslator.dispatch_visit calling visit_reference for Utilities SphinxTranslator.dispatch_visit calling visit_Text for Utilities SphinxTranslator.dispatch_departure calling depart_Text for Utilities SphinxTranslator.dispatch_departure calling depart_reference for Utilities SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Utilities SphinxTranslator.dispatch_departure calling depart_list_item for Utilities SphinxTranslator.dispatch_departure calling depart_bullet_list for First level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_list_item for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_bullet_list for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_document for Library ReferenceFirst level variablesPublic functionsUtilities [app] emitting event: 'html-page-context'('reference', 'page.html', {'parents': [], 'prev': {'link': '../tutorial/', 'title': 'Tutorials'}, ' writing output... [100%] tutorial [app] emitting event: 'doctree-resolved'(>, 'tutorial') SphinxTranslator.dispatch_visit calling visit_document for Tutorials SphinxTranslator.dispatch_visit calling visit_title for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_title for Tutorials SphinxTranslator.dispatch_departure calling depart_document for Tutorials SphinxTranslator.dispatch_visit calling visit_document for
TutorialsUsing python-blosc (or just blosc, because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the zlib module included in the standard Python library.Here are some examples on how to use it. For the full documentation, please refer to the Library Reference section.Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz.
Compressing and decompressing with <literal>blosc</literal>Let’s start creating a NumPy array with 80 MB full of data:>>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytesand let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities):>>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlibbut Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default):>>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlibIt is important to note that this is quite more than the speed of a memcpy() in this machine:>>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/swhich means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises.Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib:>>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlibThe reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs.Now, let’s have a look at the compression ratios:>>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratioHere we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string:>>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib:>>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLibHere it is a plot with the different compression ratios achieved:The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data.How about decompression?:>>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlibHere we see a couple of things:All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression).The next plot summarizes the speed benchmarks above:These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation).Finally, here it is the way to discover all the internal codecs inside your Blosc package:>>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library.
Using different filtersIn the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets.Here it is an example using the SHUFFLE filter:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533Here there is another example using BITSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffleYou can also deactivate filters completely with NOSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original bufferSo you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one.
Supporting the buffer interfacepython-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray:>>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz'Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode.
Packaging NumPy arraysWant to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way:>>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/sAlthough this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time.
Compressing from a data pointerFor avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action:>>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() TrueAs you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size).On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization.blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as:>>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True)
Fine-tuning compression parametersThere are a number of parameters that affect the de/compression bandwidth achieved by blosc:The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py.
Links to external discussions on <literal>blosc</literal> optimizationSynthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles
SphinxTranslator.dispatch_visit calling visit_section for
TutorialsUsing python-blosc (or just blosc, because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the zlib module included in the standard Python library.Here are some examples on how to use it. For the full documentation, please refer to the Library Reference section.Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz.
Compressing and decompressing with <literal>blosc</literal>Let’s start creating a NumPy array with 80 MB full of data:>>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytesand let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities):>>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlibbut Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default):>>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlibIt is important to note that this is quite more than the speed of a memcpy() in this machine:>>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/swhich means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises.Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib:>>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlibThe reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs.Now, let’s have a look at the compression ratios:>>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratioHere we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string:>>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib:>>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLibHere it is a plot with the different compression ratios achieved:The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data.How about decompression?:>>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlibHere we see a couple of things:All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression).The next plot summarizes the speed benchmarks above:These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation).Finally, here it is the way to discover all the internal codecs inside your Blosc package:>>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library.
Using different filtersIn the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets.Here it is an example using the SHUFFLE filter:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533Here there is another example using BITSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffleYou can also deactivate filters completely with NOSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original bufferSo you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one.
Supporting the buffer interfacepython-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray:>>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz'Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode.
Packaging NumPy arraysWant to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way:>>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/sAlthough this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time.
Compressing from a data pointerFor avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action:>>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() TrueAs you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size).On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization.blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as:>>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True)
Fine-tuning compression parametersThere are a number of parameters that affect the de/compression bandwidth achieved by blosc:The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py.
Links to external discussions on <literal>blosc</literal> optimizationSynthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles
SphinxTranslator.dispatch_visit calling visit_title for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_title for Tutorials SphinxTranslator.dispatch_visit calling visit_paragraph for Using python-blosc (or just blosc, because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the zlib module included in the standard Python library. SphinxTranslator.dispatch_visit calling visit_Text for Using SphinxTranslator.dispatch_departure calling depart_Text for Using SphinxTranslator.dispatch_visit calling visit_literal for python-blosc SphinxTranslator.dispatch_visit calling visit_Text for python-blosc SphinxTranslator.dispatch_departure calling depart_Text for python-blosc SphinxTranslator.dispatch_departure calling depart_literal for python-blosc SphinxTranslator.dispatch_visit calling visit_Text for (or just SphinxTranslator.dispatch_departure calling depart_Text for (or just SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for , because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the SphinxTranslator.dispatch_departure calling depart_Text for , because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the SphinxTranslator.dispatch_visit calling visit_literal for zlib SphinxTranslator.dispatch_visit calling visit_Text for zlib SphinxTranslator.dispatch_departure calling depart_Text for zlib SphinxTranslator.dispatch_departure calling depart_literal for zlib SphinxTranslator.dispatch_visit calling visit_Text for module included in the standard Python library. SphinxTranslator.dispatch_departure calling depart_Text for module included in the standard Python library. SphinxTranslator.dispatch_departure calling depart_paragraph for Using python-blosc (or just blosc, because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the zlib module included in the standard Python library. SphinxTranslator.dispatch_visit calling visit_paragraph for Here are some examples on how to use it. For the full documentation, please refer to the Library Reference section. SphinxTranslator.dispatch_visit calling visit_Text for Here are some examples on how to use it. For the full documentation, please refer to the SphinxTranslator.dispatch_departure calling depart_Text for Here are some examples on how to use it. For the full documentation, please refer to the SphinxTranslator.dispatch_visit calling visit_reference for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_reference for Library Reference SphinxTranslator.dispatch_visit calling visit_target for SphinxTranslator.dispatch_departure calling depart_target for SphinxTranslator.dispatch_visit calling visit_Text for section. SphinxTranslator.dispatch_departure calling depart_Text for section. SphinxTranslator.dispatch_departure calling depart_paragraph for Here are some examples on how to use it. For the full documentation, please refer to the Library Reference section. SphinxTranslator.dispatch_visit calling visit_paragraph for Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz. SphinxTranslator.dispatch_visit calling visit_Text for Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz. SphinxTranslator.dispatch_departure calling depart_Text for Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz. SphinxTranslator.dispatch_departure calling depart_paragraph for Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz. SphinxTranslator.dispatch_visit calling visit_section for
Compressing and decompressing with <literal>blosc</literal>Let’s start creating a NumPy array with 80 MB full of data:>>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytesand let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities):>>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlibbut Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default):>>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlibIt is important to note that this is quite more than the speed of a memcpy() in this machine:>>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/swhich means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises.Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib:>>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlibThe reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs.Now, let’s have a look at the compression ratios:>>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratioHere we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string:>>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib:>>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLibHere it is a plot with the different compression ratios achieved:The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data.How about decompression?:>>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlibHere we see a couple of things:All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression).The next plot summarizes the speed benchmarks above:These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation).Finally, here it is the way to discover all the internal codecs inside your Blosc package:>>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library.
SphinxTranslator.dispatch_visit calling visit_title for Compressing and decompressing with <literal>blosc</literal> SphinxTranslator.dispatch_visit calling visit_Text for Compressing and decompressing with SphinxTranslator.dispatch_departure calling depart_Text for Compressing and decompressing with SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_departure calling depart_title for Compressing and decompressing with <literal>blosc</literal> SphinxTranslator.dispatch_visit calling visit_paragraph for Let’s start creating a NumPy array with 80 MB full of data: SphinxTranslator.dispatch_visit calling visit_Text for Let’s start creating a NumPy array with 80 MB full of data: SphinxTranslator.dispatch_departure calling depart_Text for Let’s start creating a NumPy array with 80 MB full of data: SphinxTranslator.dispatch_departure calling depart_paragraph for Let’s start creating a NumPy array with 80 MB full of data: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytes SphinxTranslator.dispatch_visit calling visit_paragraph for and let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities): SphinxTranslator.dispatch_visit calling visit_Text for and let’s compare Blosc operation with SphinxTranslator.dispatch_departure calling depart_Text for and let’s compare Blosc operation with SphinxTranslator.dispatch_visit calling visit_literal for zlib SphinxTranslator.dispatch_visit calling visit_Text for zlib SphinxTranslator.dispatch_departure calling depart_Text for zlib SphinxTranslator.dispatch_departure calling depart_literal for zlib SphinxTranslator.dispatch_visit calling visit_Text for (please note that we are using IPython for leveraging its timing capabilities): SphinxTranslator.dispatch_departure calling depart_Text for (please note that we are using IPython for leveraging its timing capabilities): SphinxTranslator.dispatch_departure calling depart_paragraph for and let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities): SphinxTranslator.dispatch_visit calling visit_literal_block for >>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlib SphinxTranslator.dispatch_visit calling visit_paragraph for but Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default): SphinxTranslator.dispatch_visit calling visit_Text for but Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default): SphinxTranslator.dispatch_departure calling depart_Text for but Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default): SphinxTranslator.dispatch_departure calling depart_paragraph for but Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default): SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlib SphinxTranslator.dispatch_visit calling visit_paragraph for It is important to note that this is quite more than the speed of a memcpy() in this machine: SphinxTranslator.dispatch_visit calling visit_Text for It is important to note that this is quite more than the speed of a memcpy() in this machine: SphinxTranslator.dispatch_departure calling depart_Text for It is important to note that this is quite more than the speed of a memcpy() in this machine: SphinxTranslator.dispatch_departure calling depart_paragraph for It is important to note that this is quite more than the speed of a memcpy() in this machine: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/s SphinxTranslator.dispatch_visit calling visit_paragraph for which means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises. SphinxTranslator.dispatch_visit calling visit_Text for which means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises. SphinxTranslator.dispatch_departure calling depart_Text for which means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises. SphinxTranslator.dispatch_departure calling depart_paragraph for which means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises. SphinxTranslator.dispatch_visit calling visit_paragraph for Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib: SphinxTranslator.dispatch_visit calling visit_Text for Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib: SphinxTranslator.dispatch_departure calling depart_Text for Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib: SphinxTranslator.dispatch_departure calling depart_paragraph for Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlib SphinxTranslator.dispatch_visit calling visit_paragraph for The reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs. SphinxTranslator.dispatch_visit calling visit_Text for The reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs. SphinxTranslator.dispatch_departure calling depart_Text for The reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs. SphinxTranslator.dispatch_departure calling depart_paragraph for The reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs. SphinxTranslator.dispatch_visit calling visit_paragraph for Now, let’s have a look at the compression ratios: SphinxTranslator.dispatch_visit calling visit_Text for Now, let’s have a look at the compression ratios: SphinxTranslator.dispatch_departure calling depart_Text for Now, let’s have a look at the compression ratios: SphinxTranslator.dispatch_departure calling depart_paragraph for Now, let’s have a look at the compression ratios: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratio SphinxTranslator.dispatch_visit calling visit_paragraph for Here we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string: SphinxTranslator.dispatch_visit calling visit_Text for Here we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string: SphinxTranslator.dispatch_departure calling depart_Text for Here we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string: SphinxTranslator.dispatch_departure calling depart_paragraph for Here we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996 SphinxTranslator.dispatch_visit calling visit_paragraph for That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib: SphinxTranslator.dispatch_visit calling visit_Text for That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib: SphinxTranslator.dispatch_departure calling depart_Text for That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib: SphinxTranslator.dispatch_departure calling depart_paragraph for That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLib SphinxTranslator.dispatch_visit calling visit_paragraph for Here it is a plot with the different compression ratios achieved: SphinxTranslator.dispatch_visit calling visit_Text for Here it is a plot with the different compression ratios achieved: SphinxTranslator.dispatch_departure calling depart_Text for Here it is a plot with the different compression ratios achieved: SphinxTranslator.dispatch_departure calling depart_paragraph for Here it is a plot with the different compression ratios achieved: SphinxTranslator.dispatch_visit calling visit_reference for SphinxTranslator.dispatch_visit calling visit_image for SphinxTranslator.dispatch_departure calling depart_image for SphinxTranslator.dispatch_departure calling depart_reference for SphinxTranslator.dispatch_visit calling visit_paragraph for The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data. SphinxTranslator.dispatch_visit calling visit_Text for The explanation for this apparently shocking result is that Blosc uses filters ( SphinxTranslator.dispatch_departure calling depart_Text for The explanation for this apparently shocking result is that Blosc uses filters ( SphinxTranslator.dispatch_visit calling visit_literal for SHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for SHUFFLE SphinxTranslator.dispatch_departure calling depart_Text for SHUFFLE SphinxTranslator.dispatch_departure calling depart_literal for SHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for BITSHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for BITSHUFFLE SphinxTranslator.dispatch_departure calling depart_Text for BITSHUFFLE SphinxTranslator.dispatch_departure calling depart_literal for BITSHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data. SphinxTranslator.dispatch_departure calling depart_Text for currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data. SphinxTranslator.dispatch_departure calling depart_paragraph for The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data. SphinxTranslator.dispatch_visit calling visit_paragraph for How about decompression?: SphinxTranslator.dispatch_visit calling visit_Text for How about decompression?: SphinxTranslator.dispatch_departure calling depart_Text for How about decompression?: SphinxTranslator.dispatch_departure calling depart_paragraph for How about decompression?: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlib SphinxTranslator.dispatch_visit calling visit_paragraph for Here we see a couple of things: SphinxTranslator.dispatch_visit calling visit_Text for Here we see a couple of things: SphinxTranslator.dispatch_departure calling depart_Text for Here we see a couple of things: SphinxTranslator.dispatch_departure calling depart_paragraph for Here we see a couple of things: SphinxTranslator.dispatch_visit calling visit_bullet_list for All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_visit calling visit_list_item for All the internal codecs in Blosc are way faster than naked ZLib SphinxTranslator.dispatch_visit calling visit_paragraph for All the internal codecs in Blosc are way faster than naked ZLib SphinxTranslator.dispatch_visit calling visit_Text for All the internal codecs in Blosc are way faster than naked ZLib SphinxTranslator.dispatch_departure calling depart_Text for All the internal codecs in Blosc are way faster than naked ZLib SphinxTranslator.dispatch_departure calling depart_paragraph for All the internal codecs in Blosc are way faster than naked ZLib SphinxTranslator.dispatch_departure calling depart_list_item for All the internal codecs in Blosc are way faster than naked ZLib SphinxTranslator.dispatch_visit calling visit_list_item for The fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_visit calling visit_paragraph for The fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_visit calling visit_Text for The fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_departure calling depart_Text for The fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_departure calling depart_paragraph for The fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_departure calling depart_list_item for The fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_departure calling depart_bullet_list for All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_visit calling visit_paragraph for The next plot summarizes the speed benchmarks above: SphinxTranslator.dispatch_visit calling visit_Text for The next plot summarizes the speed benchmarks above: SphinxTranslator.dispatch_departure calling depart_Text for The next plot summarizes the speed benchmarks above: SphinxTranslator.dispatch_departure calling depart_paragraph for The next plot summarizes the speed benchmarks above: SphinxTranslator.dispatch_visit calling visit_reference for SphinxTranslator.dispatch_visit calling visit_image for SphinxTranslator.dispatch_departure calling depart_image for SphinxTranslator.dispatch_departure calling depart_reference for SphinxTranslator.dispatch_visit calling visit_paragraph for These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation). SphinxTranslator.dispatch_visit calling visit_Text for These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation). SphinxTranslator.dispatch_departure calling depart_Text for These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation). SphinxTranslator.dispatch_departure calling depart_paragraph for These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation). SphinxTranslator.dispatch_visit calling visit_paragraph for Finally, here it is the way to discover all the internal codecs inside your Blosc package: SphinxTranslator.dispatch_visit calling visit_Text for Finally, here it is the way to discover all the internal codecs inside your Blosc package: SphinxTranslator.dispatch_departure calling depart_Text for Finally, here it is the way to discover all the internal codecs inside your Blosc package: SphinxTranslator.dispatch_departure calling depart_paragraph for Finally, here it is the way to discover all the internal codecs inside your Blosc package: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd'] SphinxTranslator.dispatch_visit calling visit_paragraph for Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library. SphinxTranslator.dispatch_visit calling visit_emphasis for Note SphinxTranslator.dispatch_visit calling visit_Text for Note SphinxTranslator.dispatch_departure calling depart_Text for Note SphinxTranslator.dispatch_departure calling depart_emphasis for Note SphinxTranslator.dispatch_visit calling visit_Text for : the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library. SphinxTranslator.dispatch_departure calling depart_Text for : the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library. SphinxTranslator.dispatch_departure calling depart_paragraph for Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library. SphinxTranslator.dispatch_departure calling depart_section for
Compressing and decompressing with <literal>blosc</literal>Let’s start creating a NumPy array with 80 MB full of data:>>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytesand let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities):>>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlibbut Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default):>>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlibIt is important to note that this is quite more than the speed of a memcpy() in this machine:>>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/swhich means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises.Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib:>>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlibThe reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs.Now, let’s have a look at the compression ratios:>>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratioHere we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string:>>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib:>>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLibHere it is a plot with the different compression ratios achieved:The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data.How about decompression?:>>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlibHere we see a couple of things:All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression).The next plot summarizes the speed benchmarks above:These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation).Finally, here it is the way to discover all the internal codecs inside your Blosc package:>>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library.
SphinxTranslator.dispatch_visit calling visit_section for
Using different filtersIn the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets.Here it is an example using the SHUFFLE filter:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533Here there is another example using BITSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffleYou can also deactivate filters completely with NOSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original bufferSo you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one.
SphinxTranslator.dispatch_visit calling visit_title for Using different filters SphinxTranslator.dispatch_visit calling visit_Text for Using different filters SphinxTranslator.dispatch_departure calling depart_Text for Using different filters SphinxTranslator.dispatch_departure calling depart_title for Using different filters SphinxTranslator.dispatch_visit calling visit_paragraph for In the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets. SphinxTranslator.dispatch_visit calling visit_Text for In the same way that you can use different codecs inside Blosc, you can use different filters too (currently SphinxTranslator.dispatch_departure calling depart_Text for In the same way that you can use different codecs inside Blosc, you can use different filters too (currently SphinxTranslator.dispatch_visit calling visit_literal for SHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for SHUFFLE SphinxTranslator.dispatch_departure calling depart_Text for SHUFFLE SphinxTranslator.dispatch_departure calling depart_literal for SHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for BITSHUFLE SphinxTranslator.dispatch_visit calling visit_Text for BITSHUFLE SphinxTranslator.dispatch_departure calling depart_Text for BITSHUFLE SphinxTranslator.dispatch_departure calling depart_literal for BITSHUFLE SphinxTranslator.dispatch_visit calling visit_Text for ). These allow the integrated compressors to compress more efficiently or not, depending on your datasets. SphinxTranslator.dispatch_departure calling depart_Text for ). These allow the integrated compressors to compress more efficiently or not, depending on your datasets. SphinxTranslator.dispatch_departure calling depart_paragraph for In the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets. SphinxTranslator.dispatch_visit calling visit_paragraph for Here it is an example using the SHUFFLE filter: SphinxTranslator.dispatch_visit calling visit_Text for Here it is an example using the SphinxTranslator.dispatch_departure calling depart_Text for Here it is an example using the SphinxTranslator.dispatch_visit calling visit_literal for SHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for SHUFFLE SphinxTranslator.dispatch_departure calling depart_Text for SHUFFLE SphinxTranslator.dispatch_departure calling depart_literal for SHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for filter: SphinxTranslator.dispatch_departure calling depart_Text for filter: SphinxTranslator.dispatch_departure calling depart_paragraph for Here it is an example using the SHUFFLE filter: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533 SphinxTranslator.dispatch_visit calling visit_paragraph for Here there is another example using BITSHUFFLE: SphinxTranslator.dispatch_visit calling visit_Text for Here there is another example using SphinxTranslator.dispatch_departure calling depart_Text for Here there is another example using SphinxTranslator.dispatch_visit calling visit_literal for BITSHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for BITSHUFFLE SphinxTranslator.dispatch_departure calling depart_Text for BITSHUFFLE SphinxTranslator.dispatch_departure calling depart_literal for BITSHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for : SphinxTranslator.dispatch_departure calling depart_Text for : SphinxTranslator.dispatch_departure calling depart_paragraph for Here there is another example using BITSHUFFLE: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffle SphinxTranslator.dispatch_visit calling visit_paragraph for You can also deactivate filters completely with NOSHUFFLE: SphinxTranslator.dispatch_visit calling visit_Text for You can also deactivate filters completely with SphinxTranslator.dispatch_departure calling depart_Text for You can also deactivate filters completely with SphinxTranslator.dispatch_visit calling visit_literal for NOSHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for NOSHUFFLE SphinxTranslator.dispatch_departure calling depart_Text for NOSHUFFLE SphinxTranslator.dispatch_departure calling depart_literal for NOSHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for : SphinxTranslator.dispatch_departure calling depart_Text for : SphinxTranslator.dispatch_departure calling depart_paragraph for You can also deactivate filters completely with NOSHUFFLE: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original buffer SphinxTranslator.dispatch_visit calling visit_paragraph for So you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one. SphinxTranslator.dispatch_visit calling visit_Text for So you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one. SphinxTranslator.dispatch_departure calling depart_Text for So you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one. SphinxTranslator.dispatch_departure calling depart_paragraph for So you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one. SphinxTranslator.dispatch_departure calling depart_section for
Using different filtersIn the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets.Here it is an example using the SHUFFLE filter:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533Here there is another example using BITSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffleYou can also deactivate filters completely with NOSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original bufferSo you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one.
SphinxTranslator.dispatch_visit calling visit_section for
Supporting the buffer interfacepython-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray:>>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz'Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode.
SphinxTranslator.dispatch_visit calling visit_title for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_Text for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_Text for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_title for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_paragraph for python-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray: SphinxTranslator.dispatch_visit calling visit_Text for python-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes SphinxTranslator.dispatch_departure calling depart_Text for python-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes SphinxTranslator.dispatch_visit calling visit_literal for buffer SphinxTranslator.dispatch_visit calling visit_Text for buffer SphinxTranslator.dispatch_departure calling depart_Text for buffer SphinxTranslator.dispatch_departure calling depart_literal for buffer SphinxTranslator.dispatch_visit calling visit_Text for , SphinxTranslator.dispatch_departure calling depart_Text for , SphinxTranslator.dispatch_visit calling visit_literal for memoryview SphinxTranslator.dispatch_visit calling visit_Text for memoryview SphinxTranslator.dispatch_departure calling depart_Text for memoryview SphinxTranslator.dispatch_departure calling depart_literal for memoryview SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for bytearray SphinxTranslator.dispatch_visit calling visit_Text for bytearray SphinxTranslator.dispatch_departure calling depart_Text for bytearray SphinxTranslator.dispatch_departure calling depart_literal for bytearray SphinxTranslator.dispatch_visit calling visit_Text for : SphinxTranslator.dispatch_departure calling depart_Text for : SphinxTranslator.dispatch_departure calling depart_paragraph for python-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz' SphinxTranslator.dispatch_visit calling visit_paragraph for Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode. SphinxTranslator.dispatch_visit calling visit_Text for Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both SphinxTranslator.dispatch_departure calling depart_Text for Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both SphinxTranslator.dispatch_visit calling visit_literal for str SphinxTranslator.dispatch_visit calling visit_Text for str SphinxTranslator.dispatch_departure calling depart_Text for str SphinxTranslator.dispatch_departure calling depart_literal for str SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for unicode SphinxTranslator.dispatch_visit calling visit_Text for unicode SphinxTranslator.dispatch_departure calling depart_Text for unicode SphinxTranslator.dispatch_departure calling depart_literal for unicode SphinxTranslator.dispatch_visit calling visit_Text for types, whereas in Python 3.x we can only compress ‘binary’ data which does SphinxTranslator.dispatch_departure calling depart_Text for types, whereas in Python 3.x we can only compress ‘binary’ data which does SphinxTranslator.dispatch_visit calling visit_emphasis for not SphinxTranslator.dispatch_visit calling visit_Text for not SphinxTranslator.dispatch_departure calling depart_Text for not SphinxTranslator.dispatch_departure calling depart_emphasis for not SphinxTranslator.dispatch_visit calling visit_Text for include SphinxTranslator.dispatch_departure calling depart_Text for include SphinxTranslator.dispatch_visit calling visit_literal for unicode SphinxTranslator.dispatch_visit calling visit_Text for unicode SphinxTranslator.dispatch_departure calling depart_Text for unicode SphinxTranslator.dispatch_departure calling depart_literal for unicode SphinxTranslator.dispatch_visit calling visit_Text for . SphinxTranslator.dispatch_departure calling depart_Text for . SphinxTranslator.dispatch_departure calling depart_paragraph for Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode. SphinxTranslator.dispatch_departure calling depart_section for
Supporting the buffer interfacepython-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray:>>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz'Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode.
SphinxTranslator.dispatch_visit calling visit_section for
Packaging NumPy arraysWant to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way:>>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/sAlthough this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time.
SphinxTranslator.dispatch_visit calling visit_title for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_Text for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_Text for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_title for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_paragraph for Want to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way: SphinxTranslator.dispatch_visit calling visit_Text for Want to use SphinxTranslator.dispatch_departure calling depart_Text for Want to use SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? SphinxTranslator.dispatch_departure calling depart_Text for to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for comes with the SphinxTranslator.dispatch_departure calling depart_Text for comes with the SphinxTranslator.dispatch_visit calling visit_literal for pack_array SphinxTranslator.dispatch_visit calling visit_Text for pack_array SphinxTranslator.dispatch_departure calling depart_Text for pack_array SphinxTranslator.dispatch_departure calling depart_literal for pack_array SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for unpack_array SphinxTranslator.dispatch_visit calling visit_Text for unpack_array SphinxTranslator.dispatch_departure calling depart_Text for unpack_array SphinxTranslator.dispatch_departure calling depart_literal for unpack_array SphinxTranslator.dispatch_visit calling visit_Text for to perform this in a handy way: SphinxTranslator.dispatch_departure calling depart_Text for to perform this in a handy way: SphinxTranslator.dispatch_departure calling depart_paragraph for Want to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/s SphinxTranslator.dispatch_visit calling visit_paragraph for Although this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time. SphinxTranslator.dispatch_visit calling visit_Text for Although this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time. SphinxTranslator.dispatch_departure calling depart_Text for Although this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time. SphinxTranslator.dispatch_departure calling depart_paragraph for Although this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time. SphinxTranslator.dispatch_departure calling depart_section for
Packaging NumPy arraysWant to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way:>>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/sAlthough this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time.
SphinxTranslator.dispatch_visit calling visit_section for
Compressing from a data pointerFor avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action:>>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() TrueAs you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size).On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization.blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as:>>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True)
SphinxTranslator.dispatch_visit calling visit_title for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_Text for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_Text for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_title for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_paragraph for For avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action: SphinxTranslator.dispatch_visit calling visit_Text for For avoiding the data copy problem in the previous section, SphinxTranslator.dispatch_departure calling depart_Text for For avoiding the data copy problem in the previous section, SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for comes with a couple of lower-level functions: SphinxTranslator.dispatch_departure calling depart_Text for comes with a couple of lower-level functions: SphinxTranslator.dispatch_visit calling visit_literal for compress_ptr SphinxTranslator.dispatch_visit calling visit_Text for compress_ptr SphinxTranslator.dispatch_departure calling depart_Text for compress_ptr SphinxTranslator.dispatch_departure calling depart_literal for compress_ptr SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for decompress_ptr SphinxTranslator.dispatch_visit calling visit_Text for decompress_ptr SphinxTranslator.dispatch_departure calling depart_Text for decompress_ptr SphinxTranslator.dispatch_departure calling depart_literal for decompress_ptr SphinxTranslator.dispatch_visit calling visit_Text for . Here are they in action: SphinxTranslator.dispatch_departure calling depart_Text for . Here are they in action: SphinxTranslator.dispatch_departure calling depart_paragraph for For avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() True SphinxTranslator.dispatch_visit calling visit_paragraph for As you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size). SphinxTranslator.dispatch_visit calling visit_Text for As you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size). SphinxTranslator.dispatch_departure calling depart_Text for As you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size). SphinxTranslator.dispatch_departure calling depart_paragraph for As you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size). SphinxTranslator.dispatch_visit calling visit_paragraph for On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization. SphinxTranslator.dispatch_visit calling visit_Text for On the other hand, and contrarily to the SphinxTranslator.dispatch_departure calling depart_Text for On the other hand, and contrarily to the SphinxTranslator.dispatch_visit calling visit_literal for pack_array SphinxTranslator.dispatch_visit calling visit_Text for pack_array SphinxTranslator.dispatch_departure calling depart_Text for pack_array SphinxTranslator.dispatch_departure calling depart_literal for pack_array SphinxTranslator.dispatch_visit calling visit_Text for / SphinxTranslator.dispatch_departure calling depart_Text for / SphinxTranslator.dispatch_visit calling visit_literal for unpack_array SphinxTranslator.dispatch_visit calling visit_Text for unpack_array SphinxTranslator.dispatch_departure calling depart_Text for unpack_array SphinxTranslator.dispatch_departure calling depart_literal for unpack_array SphinxTranslator.dispatch_visit calling visit_Text for method, the SphinxTranslator.dispatch_departure calling depart_Text for method, the SphinxTranslator.dispatch_visit calling visit_literal for compress_ptr SphinxTranslator.dispatch_visit calling visit_Text for compress_ptr SphinxTranslator.dispatch_departure calling depart_Text for compress_ptr SphinxTranslator.dispatch_departure calling depart_literal for compress_ptr SphinxTranslator.dispatch_visit calling visit_Text for / SphinxTranslator.dispatch_departure calling depart_Text for / SphinxTranslator.dispatch_visit calling visit_literal for decompress_ptr SphinxTranslator.dispatch_visit calling visit_Text for decompress_ptr SphinxTranslator.dispatch_departure calling depart_Text for decompress_ptr SphinxTranslator.dispatch_departure calling depart_literal for decompress_ptr SphinxTranslator.dispatch_visit calling visit_Text for functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization. SphinxTranslator.dispatch_departure calling depart_Text for functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization. SphinxTranslator.dispatch_departure calling depart_paragraph for On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization. SphinxTranslator.dispatch_visit calling visit_paragraph for blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as: SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for has a maximum SphinxTranslator.dispatch_departure calling depart_Text for has a maximum SphinxTranslator.dispatch_visit calling visit_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for blocksize SphinxTranslator.dispatch_departure calling depart_Text for blocksize SphinxTranslator.dispatch_departure calling depart_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for of 2**31 bytes = 2 GB. Larger SphinxTranslator.dispatch_departure calling depart_Text for of 2**31 bytes = 2 GB. Larger SphinxTranslator.dispatch_visit calling visit_literal for numpy SphinxTranslator.dispatch_visit calling visit_Text for numpy SphinxTranslator.dispatch_departure calling depart_Text for numpy SphinxTranslator.dispatch_departure calling depart_literal for numpy SphinxTranslator.dispatch_visit calling visit_Text for arrays must be chunked by slicing, for example as: SphinxTranslator.dispatch_departure calling depart_Text for arrays must be chunked by slicing, for example as: SphinxTranslator.dispatch_departure calling depart_paragraph for blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) SphinxTranslator.dispatch_departure calling depart_section for
Compressing from a data pointerFor avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action:>>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() TrueAs you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size).On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization.blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as:>>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True)
SphinxTranslator.dispatch_visit calling visit_section for
Fine-tuning compression parametersThere are a number of parameters that affect the de/compression bandwidth achieved by blosc:The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py.
SphinxTranslator.dispatch_visit calling visit_title for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_Text for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_Text for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_title for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_paragraph for There are a number of parameters that affect the de/compression bandwidth achieved by blosc: SphinxTranslator.dispatch_visit calling visit_Text for There are a number of parameters that affect the de/compression bandwidth achieved by SphinxTranslator.dispatch_departure calling depart_Text for There are a number of parameters that affect the de/compression bandwidth achieved by SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for : SphinxTranslator.dispatch_departure calling depart_Text for : SphinxTranslator.dispatch_departure calling depart_paragraph for There are a number of parameters that affect the de/compression bandwidth achieved by blosc: SphinxTranslator.dispatch_visit calling visit_bullet_list for The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py. SphinxTranslator.dispatch_visit calling visit_list_item for The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module. SphinxTranslator.dispatch_visit calling visit_paragraph for The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module. SphinxTranslator.dispatch_visit calling visit_Text for The information content of the underlying data, and SphinxTranslator.dispatch_departure calling depart_Text for The information content of the underlying data, and SphinxTranslator.dispatch_visit calling visit_literal for chunksize SphinxTranslator.dispatch_visit calling visit_Text for chunksize SphinxTranslator.dispatch_departure calling depart_Text for chunksize SphinxTranslator.dispatch_departure calling depart_literal for chunksize SphinxTranslator.dispatch_visit calling visit_Text for : the size of the data in each call to SphinxTranslator.dispatch_departure calling depart_Text for : the size of the data in each call to SphinxTranslator.dispatch_visit calling visit_literal for compress SphinxTranslator.dispatch_visit calling visit_Text for compress SphinxTranslator.dispatch_departure calling depart_Text for compress SphinxTranslator.dispatch_departure calling depart_literal for compress SphinxTranslator.dispatch_visit calling visit_Text for and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the SphinxTranslator.dispatch_departure calling depart_Text for and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the SphinxTranslator.dispatch_visit calling visit_literal for bloscpack SphinxTranslator.dispatch_visit calling visit_Text for bloscpack SphinxTranslator.dispatch_departure calling depart_Text for bloscpack SphinxTranslator.dispatch_departure calling depart_literal for bloscpack SphinxTranslator.dispatch_visit calling visit_Text for module. SphinxTranslator.dispatch_departure calling depart_Text for module. SphinxTranslator.dispatch_departure calling depart_paragraph for The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module. SphinxTranslator.dispatch_departure calling depart_list_item for The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module. SphinxTranslator.dispatch_visit calling visit_list_item for n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores. SphinxTranslator.dispatch_visit calling visit_paragraph for n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores. SphinxTranslator.dispatch_visit calling visit_literal for n_threads SphinxTranslator.dispatch_visit calling visit_Text for n_threads SphinxTranslator.dispatch_departure calling depart_Text for n_threads SphinxTranslator.dispatch_departure calling depart_literal for n_threads SphinxTranslator.dispatch_visit calling visit_Text for : The number of threads to spawn inside SphinxTranslator.dispatch_departure calling depart_Text for : The number of threads to spawn inside SphinxTranslator.dispatch_visit calling visit_literal for c-blosc SphinxTranslator.dispatch_visit calling visit_Text for c-blosc SphinxTranslator.dispatch_departure calling depart_Text for c-blosc SphinxTranslator.dispatch_departure calling depart_literal for c-blosc SphinxTranslator.dispatch_visit calling visit_Text for . SphinxTranslator.dispatch_departure calling depart_Text for . SphinxTranslator.dispatch_visit calling visit_literal for n_threads SphinxTranslator.dispatch_visit calling visit_Text for n_threads SphinxTranslator.dispatch_departure calling depart_Text for n_threads SphinxTranslator.dispatch_departure calling depart_literal for n_threads SphinxTranslator.dispatch_visit calling visit_Text for may be changed by calling SphinxTranslator.dispatch_departure calling depart_Text for may be changed by calling SphinxTranslator.dispatch_visit calling visit_literal for blosc.set_nthreads(16) SphinxTranslator.dispatch_visit calling visit_Text for blosc.set_nthreads(16) SphinxTranslator.dispatch_departure calling depart_Text for blosc.set_nthreads(16) SphinxTranslator.dispatch_departure calling depart_literal for blosc.set_nthreads(16) SphinxTranslator.dispatch_visit calling visit_Text for for example. SphinxTranslator.dispatch_departure calling depart_Text for for example. SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores. SphinxTranslator.dispatch_departure calling depart_Text for performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores. SphinxTranslator.dispatch_departure calling depart_paragraph for n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores. SphinxTranslator.dispatch_departure calling depart_list_item for n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores. SphinxTranslator.dispatch_visit calling visit_list_item for blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization. SphinxTranslator.dispatch_visit calling visit_paragraph for blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization. SphinxTranslator.dispatch_visit calling visit_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for blocksize SphinxTranslator.dispatch_departure calling depart_Text for blocksize SphinxTranslator.dispatch_departure calling depart_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for : is the size of each continuously memory-element that is compressed, in bytes. Normally SphinxTranslator.dispatch_departure calling depart_Text for : is the size of each continuously memory-element that is compressed, in bytes. Normally SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling SphinxTranslator.dispatch_departure calling depart_Text for attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling SphinxTranslator.dispatch_visit calling visit_literal for blosc.set_blocksize( size_in_bytes ) SphinxTranslator.dispatch_visit calling visit_Text for blosc.set_blocksize( size_in_bytes ) SphinxTranslator.dispatch_departure calling depart_Text for blosc.set_blocksize( size_in_bytes ) SphinxTranslator.dispatch_departure calling depart_literal for blosc.set_blocksize( size_in_bytes ) SphinxTranslator.dispatch_visit calling visit_Text for for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization. SphinxTranslator.dispatch_departure calling depart_Text for for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization. SphinxTranslator.dispatch_departure calling depart_paragraph for blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization. SphinxTranslator.dispatch_departure calling depart_list_item for blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization. SphinxTranslator.dispatch_visit calling visit_list_item for shuffle: as discussed above the shuffle mode can substantially improve compression ratios when the SphinxTranslator.dispatch_visit calling visit_paragraph for shuffle: as discussed above the shuffle mode can substantially improve compression ratios when the SphinxTranslator.dispatch_visit calling visit_literal for shuffle SphinxTranslator.dispatch_visit calling visit_Text for shuffle SphinxTranslator.dispatch_departure calling depart_Text for shuffle SphinxTranslator.dispatch_departure calling depart_literal for shuffle SphinxTranslator.dispatch_visit calling visit_Text for : as discussed above the SphinxTranslator.dispatch_departure calling depart_Text for : as discussed above the SphinxTranslator.dispatch_visit calling visit_literal for shuffle SphinxTranslator.dispatch_visit calling visit_Text for shuffle SphinxTranslator.dispatch_departure calling depart_Text for shuffle SphinxTranslator.dispatch_departure calling depart_literal for shuffle SphinxTranslator.dispatch_visit calling visit_Text for mode can substantially improve compression ratios when the SphinxTranslator.dispatch_departure calling depart_Text for mode can substantially improve compression ratios when the SphinxTranslator.dispatch_departure calling depart_paragraph for shuffle: as discussed above the shuffle mode can substantially improve compression ratios when the SphinxTranslator.dispatch_departure calling depart_list_item for shuffle: as discussed above the shuffle mode can substantially improve compression ratios when the SphinxTranslator.dispatch_visit calling visit_list_item for clevel: the compression level called for the algorithm. Called as an argument to compress and similar functions. SphinxTranslator.dispatch_visit calling visit_paragraph for clevel: the compression level called for the algorithm. Called as an argument to compress and similar functions. SphinxTranslator.dispatch_visit calling visit_literal for clevel SphinxTranslator.dispatch_visit calling visit_Text for clevel SphinxTranslator.dispatch_departure calling depart_Text for clevel SphinxTranslator.dispatch_departure calling depart_literal for clevel SphinxTranslator.dispatch_visit calling visit_Text for : the compression level called for the algorithm. Called as an argument to SphinxTranslator.dispatch_departure calling depart_Text for : the compression level called for the algorithm. Called as an argument to SphinxTranslator.dispatch_visit calling visit_literal for compress SphinxTranslator.dispatch_visit calling visit_Text for compress SphinxTranslator.dispatch_departure calling depart_Text for compress SphinxTranslator.dispatch_departure calling depart_literal for compress SphinxTranslator.dispatch_visit calling visit_Text for and similar functions. SphinxTranslator.dispatch_departure calling depart_Text for and similar functions. SphinxTranslator.dispatch_departure calling depart_paragraph for clevel: the compression level called for the algorithm. Called as an argument to compress and similar functions. SphinxTranslator.dispatch_departure calling depart_list_item for clevel: the compression level called for the algorithm. Called as an argument to compress and similar functions. SphinxTranslator.dispatch_visit calling visit_list_item for cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions. SphinxTranslator.dispatch_visit calling visit_paragraph for cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions. SphinxTranslator.dispatch_visit calling visit_literal for cname SphinxTranslator.dispatch_visit calling visit_Text for cname SphinxTranslator.dispatch_departure calling depart_Text for cname SphinxTranslator.dispatch_departure calling depart_literal for cname SphinxTranslator.dispatch_visit calling visit_Text for : the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller SphinxTranslator.dispatch_departure calling depart_Text for : the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller SphinxTranslator.dispatch_visit calling visit_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for blocksize SphinxTranslator.dispatch_departure calling depart_Text for blocksize SphinxTranslator.dispatch_departure calling depart_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for and does not slow significantly with SphinxTranslator.dispatch_departure calling depart_Text for and does not slow significantly with SphinxTranslator.dispatch_visit calling visit_literal for clevel SphinxTranslator.dispatch_visit calling visit_Text for clevel SphinxTranslator.dispatch_departure calling depart_Text for clevel SphinxTranslator.dispatch_departure calling depart_literal for clevel SphinxTranslator.dispatch_visit calling visit_Text for . In comparison SphinxTranslator.dispatch_departure calling depart_Text for . In comparison SphinxTranslator.dispatch_visit calling visit_literal for zlib SphinxTranslator.dispatch_visit calling visit_Text for zlib SphinxTranslator.dispatch_departure calling depart_Text for zlib SphinxTranslator.dispatch_departure calling depart_literal for zlib SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for zstd SphinxTranslator.dispatch_visit calling visit_Text for zstd SphinxTranslator.dispatch_departure calling depart_Text for zstd SphinxTranslator.dispatch_departure calling depart_literal for zstd SphinxTranslator.dispatch_visit calling visit_Text for both slow substantially with increasing SphinxTranslator.dispatch_departure calling depart_Text for both slow substantially with increasing SphinxTranslator.dispatch_visit calling visit_literal for clevel SphinxTranslator.dispatch_visit calling visit_Text for clevel SphinxTranslator.dispatch_departure calling depart_Text for clevel SphinxTranslator.dispatch_departure calling depart_literal for clevel SphinxTranslator.dispatch_visit calling visit_Text for , with an inflection point around SphinxTranslator.dispatch_departure calling depart_Text for , with an inflection point around SphinxTranslator.dispatch_visit calling visit_literal for clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress SphinxTranslator.dispatch_visit calling visit_Text for clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress SphinxTranslator.dispatch_departure calling depart_Text for clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress SphinxTranslator.dispatch_departure calling depart_literal for clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress SphinxTranslator.dispatch_visit calling visit_Text for and similar functions. SphinxTranslator.dispatch_departure calling depart_Text for and similar functions. SphinxTranslator.dispatch_departure calling depart_paragraph for cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions. SphinxTranslator.dispatch_departure calling depart_list_item for cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions. SphinxTranslator.dispatch_visit calling visit_list_item for releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py. SphinxTranslator.dispatch_visit calling visit_paragraph for releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py. SphinxTranslator.dispatch_visit calling visit_literal for releasegil SphinxTranslator.dispatch_visit calling visit_Text for releasegil SphinxTranslator.dispatch_departure calling depart_Text for releasegil SphinxTranslator.dispatch_departure calling depart_literal for releasegil SphinxTranslator.dispatch_visit calling visit_Text for : optionally the Python Global Interpreter Lock (GIL) can be turned off during SphinxTranslator.dispatch_departure calling depart_Text for : optionally the Python Global Interpreter Lock (GIL) can be turned off during SphinxTranslator.dispatch_visit calling visit_literal for c-blosc SphinxTranslator.dispatch_visit calling visit_Text for c-blosc SphinxTranslator.dispatch_departure calling depart_Text for c-blosc SphinxTranslator.dispatch_departure calling depart_literal for c-blosc SphinxTranslator.dispatch_visit calling visit_Text for operations by calling SphinxTranslator.dispatch_departure calling depart_Text for operations by calling SphinxTranslator.dispatch_visit calling visit_literal for blosc.set_releasegil(True) SphinxTranslator.dispatch_visit calling visit_Text for blosc.set_releasegil(True) SphinxTranslator.dispatch_departure calling depart_Text for blosc.set_releasegil(True) SphinxTranslator.dispatch_departure calling depart_literal for blosc.set_releasegil(True) SphinxTranslator.dispatch_visit calling visit_Text for . The default is off as there is a small overhead in releasing the GIL which can be significant for a small SphinxTranslator.dispatch_departure calling depart_Text for . The default is off as there is a small overhead in releasing the GIL which can be significant for a small SphinxTranslator.dispatch_visit calling visit_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for blocksize SphinxTranslator.dispatch_departure calling depart_Text for blocksize SphinxTranslator.dispatch_departure calling depart_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for . GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python SphinxTranslator.dispatch_departure calling depart_Text for . GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python SphinxTranslator.dispatch_visit calling visit_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for or similar object can be used for parallel processing either with or without SphinxTranslator.dispatch_departure calling depart_Text for or similar object can be used for parallel processing either with or without SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for threads. Initial testing suggests that an equal mix of SphinxTranslator.dispatch_departure calling depart_Text for threads. Initial testing suggests that an equal mix of SphinxTranslator.dispatch_visit calling visit_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for threads is near optimal. I.e. a computer with 16 cores would have 4 SphinxTranslator.dispatch_departure calling depart_Text for threads is near optimal. I.e. a computer with 16 cores would have 4 SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for threads and 4 SphinxTranslator.dispatch_departure calling depart_Text for threads and 4 SphinxTranslator.dispatch_visit calling visit_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for threads. An example of combining SphinxTranslator.dispatch_departure calling depart_Text for threads. An example of combining SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for may be found in SphinxTranslator.dispatch_departure calling depart_Text for may be found in SphinxTranslator.dispatch_visit calling visit_literal for bench/threadpool.py SphinxTranslator.dispatch_visit calling visit_Text for bench/threadpool.py SphinxTranslator.dispatch_departure calling depart_Text for bench/threadpool.py SphinxTranslator.dispatch_departure calling depart_literal for bench/threadpool.py SphinxTranslator.dispatch_visit calling visit_Text for . SphinxTranslator.dispatch_departure calling depart_Text for . SphinxTranslator.dispatch_departure calling depart_paragraph for releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py. SphinxTranslator.dispatch_departure calling depart_list_item for releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py. SphinxTranslator.dispatch_departure calling depart_bullet_list for The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py. SphinxTranslator.dispatch_departure calling depart_section for
Fine-tuning compression parametersThere are a number of parameters that affect the de/compression bandwidth achieved by blosc:The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py.
SphinxTranslator.dispatch_visit calling visit_section for
Links to external discussions on <literal>blosc</literal> optimizationSynthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles
SphinxTranslator.dispatch_visit calling visit_title for Links to external discussions on <literal>blosc</literal> optimization SphinxTranslator.dispatch_visit calling visit_Text for Links to external discussions on SphinxTranslator.dispatch_departure calling depart_Text for Links to external discussions on SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for optimization SphinxTranslator.dispatch_departure calling depart_Text for optimization SphinxTranslator.dispatch_departure calling depart_title for Links to external discussions on <literal>blosc</literal> optimization SphinxTranslator.dispatch_visit calling visit_bullet_list for Synthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles SphinxTranslator.dispatch_visit calling visit_list_item for Synthetic Benchmarks by Francesc Alted SphinxTranslator.dispatch_visit calling visit_paragraph for Synthetic Benchmarks by Francesc Alted SphinxTranslator.dispatch_visit calling visit_reference for Synthetic Benchmarks SphinxTranslator.dispatch_visit calling visit_Text for Synthetic Benchmarks SphinxTranslator.dispatch_departure calling depart_Text for Synthetic Benchmarks SphinxTranslator.dispatch_departure calling depart_reference for Synthetic Benchmarks SphinxTranslator.dispatch_visit calling visit_target for SphinxTranslator.dispatch_departure calling depart_target for SphinxTranslator.dispatch_visit calling visit_Text for by Francesc Alted SphinxTranslator.dispatch_departure calling depart_Text for by Francesc Alted SphinxTranslator.dispatch_departure calling depart_paragraph for Synthetic Benchmarks by Francesc Alted SphinxTranslator.dispatch_departure calling depart_list_item for Synthetic Benchmarks by Francesc Alted SphinxTranslator.dispatch_visit calling visit_list_item for Genotype compressor benchmark by Alistair Miles SphinxTranslator.dispatch_visit calling visit_paragraph for Genotype compressor benchmark by Alistair Miles SphinxTranslator.dispatch_visit calling visit_reference for Genotype compressor benchmark SphinxTranslator.dispatch_visit calling visit_Text for Genotype compressor benchmark SphinxTranslator.dispatch_departure calling depart_Text for Genotype compressor benchmark SphinxTranslator.dispatch_departure calling depart_reference for Genotype compressor benchmark SphinxTranslator.dispatch_visit calling visit_target for SphinxTranslator.dispatch_departure calling depart_target for SphinxTranslator.dispatch_visit calling visit_Text for by Alistair Miles SphinxTranslator.dispatch_departure calling depart_Text for by Alistair Miles SphinxTranslator.dispatch_departure calling depart_paragraph for Genotype compressor benchmark by Alistair Miles SphinxTranslator.dispatch_departure calling depart_list_item for Genotype compressor benchmark by Alistair Miles SphinxTranslator.dispatch_departure calling depart_bullet_list for Synthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles SphinxTranslator.dispatch_departure calling depart_section for
Links to external discussions on <literal>blosc</literal> optimizationSynthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles
SphinxTranslator.dispatch_departure calling depart_section for
TutorialsUsing python-blosc (or just blosc, because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the zlib module included in the standard Python library.Here are some examples on how to use it. For the full documentation, please refer to the Library Reference section.Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz.
Compressing and decompressing with <literal>blosc</literal>Let’s start creating a NumPy array with 80 MB full of data:>>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytesand let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities):>>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlibbut Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default):>>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlibIt is important to note that this is quite more than the speed of a memcpy() in this machine:>>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/swhich means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises.Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib:>>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlibThe reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs.Now, let’s have a look at the compression ratios:>>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratioHere we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string:>>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib:>>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLibHere it is a plot with the different compression ratios achieved:The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data.How about decompression?:>>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlibHere we see a couple of things:All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression).The next plot summarizes the speed benchmarks above:These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation).Finally, here it is the way to discover all the internal codecs inside your Blosc package:>>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library.
Using different filtersIn the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets.Here it is an example using the SHUFFLE filter:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533Here there is another example using BITSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffleYou can also deactivate filters completely with NOSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original bufferSo you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one.
Supporting the buffer interfacepython-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray:>>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz'Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode.
Packaging NumPy arraysWant to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way:>>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/sAlthough this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time.
Compressing from a data pointerFor avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action:>>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() TrueAs you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size).On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization.blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as:>>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True)
Fine-tuning compression parametersThere are a number of parameters that affect the de/compression bandwidth achieved by blosc:The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py.
Links to external discussions on <literal>blosc</literal> optimizationSynthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles
SphinxTranslator.dispatch_departure calling depart_document for
TutorialsUsing python-blosc (or just blosc, because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the zlib module included in the standard Python library.Here are some examples on how to use it. For the full documentation, please refer to the Library Reference section.Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz.
Compressing and decompressing with <literal>blosc</literal>Let’s start creating a NumPy array with 80 MB full of data:>>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytesand let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities):>>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlibbut Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default):>>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlibIt is important to note that this is quite more than the speed of a memcpy() in this machine:>>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/swhich means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises.Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib:>>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlibThe reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs.Now, let’s have a look at the compression ratios:>>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratioHere we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string:>>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib:>>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLibHere it is a plot with the different compression ratios achieved:The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data.How about decompression?:>>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlibHere we see a couple of things:All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression).The next plot summarizes the speed benchmarks above:These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation).Finally, here it is the way to discover all the internal codecs inside your Blosc package:>>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library.
Using different filtersIn the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets.Here it is an example using the SHUFFLE filter:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533Here there is another example using BITSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffleYou can also deactivate filters completely with NOSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original bufferSo you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one.
Supporting the buffer interfacepython-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray:>>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz'Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode.
Packaging NumPy arraysWant to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way:>>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/sAlthough this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time.
Compressing from a data pointerFor avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action:>>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() TrueAs you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size).On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization.blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as:>>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True)
Fine-tuning compression parametersThere are a number of parameters that affect the de/compression bandwidth achieved by blosc:The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py.
Links to external discussions on <literal>blosc</literal> optimizationSynthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles
SphinxTranslator.dispatch_visit calling visit_document for Library Reference SphinxTranslator.dispatch_visit calling visit_title for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_title for Library Reference SphinxTranslator.dispatch_departure calling depart_document for Library Reference SphinxTranslator.dispatch_visit calling visit_document for Installation SphinxTranslator.dispatch_visit calling visit_title for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_title for Installation SphinxTranslator.dispatch_departure calling depart_document for Installation SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for Tutorials SphinxTranslator.dispatch_visit calling visit_title for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_title for Tutorials SphinxTranslator.dispatch_departure calling depart_document for Tutorials SphinxTranslator.dispatch_visit calling visit_document for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_bullet_list for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_list_item for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Tutorials SphinxTranslator.dispatch_visit calling visit_reference for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_reference for Tutorials SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Tutorials SphinxTranslator.dispatch_visit calling visit_bullet_list for Compressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_list_item for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_reference for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_Text for Compressing and decompressing with SphinxTranslator.dispatch_departure calling depart_Text for Compressing and decompressing with SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_departure calling depart_reference for Compressing and decompressing with blosc SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Compressing and decompressing with blosc SphinxTranslator.dispatch_departure calling depart_list_item for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_list_item for Using different filters SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Using different filters SphinxTranslator.dispatch_visit calling visit_reference for Using different filters SphinxTranslator.dispatch_visit calling visit_Text for Using different filters SphinxTranslator.dispatch_departure calling depart_Text for Using different filters SphinxTranslator.dispatch_departure calling depart_reference for Using different filters SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Using different filters SphinxTranslator.dispatch_departure calling depart_list_item for Using different filters SphinxTranslator.dispatch_visit calling visit_list_item for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_reference for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_Text for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_Text for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_reference for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_list_item for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_list_item for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_reference for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_Text for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_Text for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_reference for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_list_item for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_list_item for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_reference for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_Text for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_Text for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_reference for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_list_item for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_list_item for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_reference for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_Text for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_Text for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_reference for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_list_item for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_list_item for Links to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Links to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_reference for Links to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_Text for Links to external discussions on SphinxTranslator.dispatch_departure calling depart_Text for Links to external discussions on SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for optimization SphinxTranslator.dispatch_departure calling depart_Text for optimization SphinxTranslator.dispatch_departure calling depart_reference for Links to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Links to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_list_item for Links to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_bullet_list for Compressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_list_item for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_bullet_list for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_document for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization [app] emitting event: 'html-page-context'('tutorial', 'page.html', {'parents': [], 'prev': {'link': '../install/', 'title': 'Installation'}, generating indices... genindex[app] emitting event: 'html-page-context'('genindex', 'genindex.html', {'genindexentries': [('_', [('__version__', [[('', '../reference/#__ve py-modindex[app] emitting event: 'html-page-context'('py-modindex', 'domainindex.html', {'indextitle': 'Python Module Index', 'content': [('b', [IndexEn done [app] emitting event: 'html-collect-pages'() writing additional pages... search[app] emitting event: 'html-page-context'('search', 'search.html', {'current_page_name': 'search', 'sidebars': ['localtoc.html', 'relations.h done copying images... [ 50%] cratio-blosc-codecs.png copying images... [100%] speed-blosc-codecs.png copying static files... ... done copying extra files... done dumping search index in English (code: en)... done dumping object inventory... done build succeeded, 5 warnings. You can now process the pickle files in _build/pickle. [app] emitting event: 'build-finished'(None,) Build finished; now you can process the pickle files. make: Leaving directory '/usr/src/RPM/BUILD/python3-module-blosc-1.5.1/doc' + make -C doc html make: Entering directory '/usr/src/RPM/BUILD/python3-module-blosc-1.5.1/doc' sphinx-build-3 -b html -d _build/doctrees -n -v -v . _build/html Running Sphinx v2.4.4 [app] setting up extension: 'sphinx.addnodes' [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] setting up extension: 'sphinx.builders.changes' [app] adding builder: [app] setting up extension: 'sphinx.builders.epub3' [app] adding builder: [app] adding config value: ('epub_basename', . at 0xf649b580>, None) [app] adding config value: ('epub_version', 3.0, 'epub') [app] adding config value: ('epub_theme', 'epub', 'epub') [app] adding config value: ('epub_theme_options', {}, 'epub') [app] adding config value: ('epub_title', . at 0xf5f22d18>, 'epub') [app] adding config value: ('epub_author', . at 0xf5f22d60>, 'epub') [app] adding config value: ('epub_language', . at 0xf5f22da8>, 'epub') [app] adding config value: ('epub_publisher', . at 0xf5f22df0>, 'epub') [app] adding config value: ('epub_copyright', . at 0xf5f22e38>, 'epub') [app] adding config value: ('epub_identifier', 'unknown', 'epub') [app] adding config value: ('epub_scheme', 'unknown', 'epub') [app] adding config value: ('epub_uid', 'unknown', 'env') [app] adding config value: ('epub_cover', (), 'env') [app] adding config value: ('epub_guide', (), 'env') [app] adding config value: ('epub_pre_files', [], 'env') [app] adding config value: ('epub_post_files', [], 'env') [app] adding config value: ('epub_css_files', . at 0xf5f22e80>, 'epub') [app] adding config value: ('epub_exclude_files', [], 'env') [app] adding config value: ('epub_tocdepth', 3, 'env') [app] adding config value: ('epub_tocdup', True, 'env') [app] adding config value: ('epub_tocscope', 'default', 'env') [app] adding config value: ('epub_fix_images', False, 'env') [app] adding config value: ('epub_max_image_width', 0, 'env') [app] adding config value: ('epub_show_urls', 'inline', 'epub') [app] adding config value: ('epub_use_index', . at 0xf5f22ec8>, 'epub') [app] adding config value: ('epub_description', 'unknown', 'epub') [app] adding config value: ('epub_contributor', 'unknown', 'epub') [app] adding config value: ('epub_writing_mode', 'horizontal', 'epub', ) [app] connecting event 'config-inited': [id=0] [app] connecting event 'builder-inited': [id=1] [app] setting up extension: 'sphinx.builders.dirhtml' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding config value: ('html_theme', 'alabaster', 'html') [app] adding config value: ('html_theme_path', [], 'html') [app] adding config value: ('html_theme_options', {}, 'html') [app] adding config value: ('html_title', . at 0xf5f22fa0>, 'html', []) [app] adding config value: ('html_short_title', . at 0xf5f26028>, 'html') [app] adding config value: ('html_style', None, 'html', []) [app] adding config value: ('html_logo', None, 'html', []) [app] adding config value: ('html_favicon', None, 'html', []) [app] adding config value: ('html_css_files', [], 'html') [app] adding config value: ('html_js_files', [], 'html') [app] adding config value: ('html_static_path', [], 'html') [app] adding config value: ('html_extra_path', [], 'html') [app] adding config value: ('html_last_updated_fmt', None, 'html', []) [app] adding config value: ('html_sidebars', {}, 'html') [app] adding config value: ('html_additional_pages', {}, 'html') [app] adding config value: ('html_domain_indices', True, 'html', []) [app] adding config value: ('html_add_permalinks', '¶', 'html') [app] adding config value: ('html_use_index', True, 'html') [app] adding config value: ('html_split_index', False, 'html') [app] adding config value: ('html_copy_source', True, 'html') [app] adding config value: ('html_show_sourcelink', True, 'html') [app] adding config value: ('html_sourcelink_suffix', '.txt', 'html') [app] adding config value: ('html_use_opensearch', '', 'html') [app] adding config value: ('html_file_suffix', None, 'html', []) [app] adding config value: ('html_link_suffix', None, 'html', []) [app] adding config value: ('html_show_copyright', True, 'html') [app] adding config value: ('html_show_sphinx', True, 'html') [app] adding config value: ('html_context', {}, 'html') [app] adding config value: ('html_output_encoding', 'utf-8', 'html') [app] adding config value: ('html_compact_lists', True, 'html') [app] adding config value: ('html_secnumber_suffix', '. ', 'html') [app] adding config value: ('html_search_language', None, 'html', []) [app] adding config value: ('html_search_options', {}, 'html') [app] adding config value: ('html_search_scorer', '', None) [app] adding config value: ('html_scaled_image_link', True, 'html') [app] adding config value: ('html_baseurl', '', 'html') [app] adding config value: ('html_math_renderer', None, 'env') [app] adding config value: ('html4_writer', False, 'html') [app] connecting event 'config-inited': [id=2] [app] connecting event 'config-inited': [id=3] [app] connecting event 'config-inited': [id=4] [app] connecting event 'config-inited': [id=5] [app] connecting event 'config-inited': [id=6] [app] connecting event 'config-inited': [id=7] [app] connecting event 'builder-inited': [id=8] [app] connecting event 'html-page-context': [id=9] [app] setting up extension: 'sphinx.ext.mathjax' [app] adding html_math_renderer: mathjax, (, None), (, None) [app] adding config value: ('mathjax_path', 'https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML', 'html') [app] adding config value: ('mathjax_options', {}, 'html') [app] adding config value: ('mathjax_inline', ['\\(', '\\)'], 'html') [app] adding config value: ('mathjax_display', ['\\[', '\\]'], 'html') [app] adding config value: ('mathjax_config', None, 'html') [app] connecting event 'env-updated': [id=10] [app] adding builder: [app] setting up extension: 'sphinx.builders.dummy' [app] adding builder: [app] setting up extension: 'sphinx.builders.gettext' [app] adding builder: [app] adding config value: ('gettext_compact', True, 'gettext') [app] adding config value: ('gettext_location', True, 'gettext') [app] adding config value: ('gettext_uuid', False, 'gettext') [app] adding config value: ('gettext_auto_build', True, 'env') [app] adding config value: ('gettext_additional_targets', [], 'env') [app] adding config value: ('gettext_last_translator', 'FULL NAME ', 'gettext') [app] adding config value: ('gettext_language_team', 'LANGUAGE ', 'gettext') [app] setting up extension: 'sphinx.builders.html' [app] setting up extension: 'sphinx.builders.latex' [app] setting up extension: 'sphinx.builders.latex.transforms' [app] adding transform: [app] adding post transform: [app] adding post transform: [app] adding post transform: [app] adding post transform: [app] adding post transform: [app] adding post transform: [app] adding post transform: [app] adding post transform: [app] adding builder: [app] connecting event 'config-inited': [id=11] [app] adding config value: ('latex_engine', , None, ) [app] adding config value: ('latex_documents', , None) [app] adding config value: ('latex_logo', None, None, []) [app] adding config value: ('latex_appendices', [], None) [app] adding config value: ('latex_use_latex_multicolumn', False, None) [app] adding config value: ('latex_use_xindy', , None, []) [app] adding config value: ('latex_toplevel_sectioning', None, None, ) [app] adding config value: ('latex_domain_indices', True, None, []) [app] adding config value: ('latex_show_urls', 'no', None) [app] adding config value: ('latex_show_pagerefs', False, None) [app] adding config value: ('latex_elements', {}, None) [app] adding config value: ('latex_additional_files', [], None) [app] adding config value: ('latex_docclass', , None) [app] setting up extension: 'sphinx.builders.linkcheck' [app] adding builder: [app] adding config value: ('linkcheck_ignore', [], None) [app] adding config value: ('linkcheck_auth', [], None) [app] adding config value: ('linkcheck_retries', 1, None) [app] adding config value: ('linkcheck_timeout', None, None, []) [app] adding config value: ('linkcheck_workers', 5, None) [app] adding config value: ('linkcheck_anchors', True, None) [app] adding config value: ('linkcheck_anchors_ignore', ['^!'], None) [app] setting up extension: 'sphinx.builders.manpage' [app] adding builder: [app] adding config value: ('man_pages', , None) [app] adding config value: ('man_show_urls', False, None) [app] setting up extension: 'sphinx.builders.singlehtml' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding config value: ('singlehtml_sidebars', . at 0xf5f260b8>, 'html') [app] setting up extension: 'sphinx.builders.texinfo' [app] adding builder: [app] adding config value: ('texinfo_documents', , None) [app] adding config value: ('texinfo_appendices', [], None) [app] adding config value: ('texinfo_elements', {}, None) [app] adding config value: ('texinfo_domain_indices', True, None, []) [app] adding config value: ('texinfo_show_urls', 'footnote', None) [app] adding config value: ('texinfo_no_detailmenu', False, None) [app] setting up extension: 'sphinx.builders.text' [app] adding builder: [app] adding config value: ('text_sectionchars', '*=-~"+`', 'env') [app] adding config value: ('text_newlines', 'unix', 'env') [app] adding config value: ('text_add_secnumbers', True, 'env') [app] adding config value: ('text_secnumber_suffix', '. ', 'env') [app] setting up extension: 'sphinx.builders.xml' [app] adding builder: [app] adding builder: [app] adding config value: ('xml_pretty', True, 'env') [app] setting up extension: 'sphinx.config' [app] connecting event 'config-inited': [id=12] [app] connecting event 'config-inited': [id=13] [app] connecting event 'config-inited': [id=14] [app] connecting event 'config-inited': [id=15] [app] connecting event 'config-inited': [id=16] [app] connecting event 'env-get-outdated': [id=17] [app] setting up extension: 'sphinx.domains.c' [app] adding domain: [app] setting up extension: 'sphinx.domains.changeset' [app] adding domain: [app] adding directive: ('deprecated', , None, None, {}) [app] adding directive: ('versionadded', , None, None, {}) [app] adding directive: ('versionchanged', , None, None, {}) [app] setting up extension: 'sphinx.domains.citation' [app] adding domain: [app] adding transform: [app] adding transform: [app] setting up extension: 'sphinx.domains.cpp' [app] adding domain: [app] adding config value: ('cpp_index_common_prefix', [], 'env') [app] adding config value: ('cpp_id_attributes', [], 'env') [app] adding config value: ('cpp_paren_attributes', [], 'env') [app] adding post transform: [app] setting up extension: 'sphinx.domains.index' [app] adding domain: [app] adding directive: ('index', , None, None, {}) [app] adding role: ('index', ) [app] setting up extension: 'sphinx.domains.javascript' [app] adding domain: [app] setting up extension: 'sphinx.domains.math' [app] adding domain: [app] adding role: ('eq', ) [app] setting up extension: 'sphinx.domains.python' [app] adding domain: [app] setting up extension: 'sphinx.domains.rst' [app] adding domain: [app] setting up extension: 'sphinx.domains.std' [app] adding domain: [app] setting up extension: 'sphinx.directives' [app] adding event: 'object-description-transform' [app] setting up extension: 'sphinx.directives.code' [app] setting up extension: 'sphinx.directives.other' [app] setting up extension: 'sphinx.directives.patches' [app] setting up extension: 'sphinx.extension' [app] connecting event 'config-inited': [id=18] [app] setting up extension: 'sphinx.parsers' [app] adding search source_parser: (,) [app] setting up extension: 'sphinx.registry' [app] connecting event 'config-inited': [id=19] [app] setting up extension: 'sphinx.roles' [app] setting up extension: 'sphinx.transforms' [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] adding transform: [app] setting up extension: 'sphinx.transforms.compact_bullet_list' [app] adding transform: [app] setting up extension: 'sphinx.transforms.i18n' [app] adding transform: [app] adding transform: [app] adding transform: [app] setting up extension: 'sphinx.transforms.references' [app] adding transform: [app] adding transform: [app] adding transform: [app] setting up extension: 'sphinx.transforms.post_transforms' [app] adding post transform: [app] adding post transform: [app] setting up extension: 'sphinx.transforms.post_transforms.code' [app] adding post transform: [app] adding post transform: [app] setting up extension: 'sphinx.transforms.post_transforms.images' [app] adding post transform: [app] adding post transform: [app] setting up extension: 'sphinx.transforms.post_transforms.compat' [app] adding post transform: [app] setting up extension: 'sphinx.util.compat' [app] adding transform: [app] connecting event 'config-inited': [id=20] [app] connecting event 'builder-inited': [id=21] [app] setting up extension: 'sphinx.versioning' [app] adding transform: [app] setting up extension: 'sphinx.environment.collectors.dependencies' [app] adding environment collector: [app] connecting event 'doctree-read': > [id=22] [app] connecting event 'env-merge-info': > [id=23] [app] connecting event 'env-purge-doc': > [id=24] [app] connecting event 'env-get-updated': > [id=25] [app] connecting event 'env-get-outdated': > [id=26] [app] setting up extension: 'sphinx.environment.collectors.asset' [app] adding environment collector: [app] connecting event 'doctree-read': > [id=27] [app] connecting event 'env-merge-info': > [id=28] [app] connecting event 'env-purge-doc': > [id=29] [app] connecting event 'env-get-updated': > [id=30] [app] connecting event 'env-get-outdated': > [id=31] [app] adding environment collector: [app] connecting event 'doctree-read': > [id=32] [app] connecting event 'env-merge-info': > [id=33] [app] connecting event 'env-purge-doc': > [id=34] [app] connecting event 'env-get-updated': > [id=35] [app] connecting event 'env-get-outdated': > [id=36] [app] setting up extension: 'sphinx.environment.collectors.metadata' [app] adding environment collector: [app] connecting event 'doctree-read': > [id=37] [app] connecting event 'env-merge-info': > [id=38] [app] connecting event 'env-purge-doc': > [id=39] [app] connecting event 'env-get-updated': > [id=40] [app] connecting event 'env-get-outdated': > [id=41] [app] setting up extension: 'sphinx.environment.collectors.title' [app] adding environment collector: [app] connecting event 'doctree-read': > [id=42] [app] connecting event 'env-merge-info': > [id=43] [app] connecting event 'env-purge-doc': > [id=44] [app] connecting event 'env-get-updated': > [id=45] [app] connecting event 'env-get-outdated': > [id=46] [app] setting up extension: 'sphinx.environment.collectors.toctree' [app] adding environment collector: [app] connecting event 'doctree-read': > [id=47] [app] connecting event 'env-merge-info': > [id=48] [app] connecting event 'env-purge-doc': > [id=49] [app] connecting event 'env-get-updated': > [id=50] [app] connecting event 'env-get-outdated': > [id=51] [app] setting up extension: 'sphinxcontrib.applehelp' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding config value: ('applehelp_bundle_name', . at 0xf5b41b20>, 'applehelp') [app] adding config value: ('applehelp_bundle_id', None, 'applehelp', []) [app] adding config value: ('applehelp_dev_region', 'en-us', 'applehelp') [app] adding config value: ('applehelp_bundle_version', '1', 'applehelp') [app] adding config value: ('applehelp_icon', None, 'applehelp', []) [app] adding config value: ('applehelp_kb_product', . at 0xf5b50538>, 'applehelp') [app] adding config value: ('applehelp_kb_url', None, 'applehelp', []) [app] adding config value: ('applehelp_remote_url', None, 'applehelp', []) [app] adding config value: ('applehelp_index_anchors', False, 'applehelp', []) [app] adding config value: ('applehelp_min_term_length', None, 'applehelp', []) [app] adding config value: ('applehelp_stopwords', . at 0xf5b504f0>, 'applehelp') [app] adding config value: ('applehelp_locale', . at 0xf5b50ec8>, 'applehelp') [app] adding config value: ('applehelp_title', . at 0xf5b50f10>, 'applehelp') [app] adding config value: ('applehelp_codesign_identity', . at 0xf5b50f58>, 'applehelp') [app] adding config value: ('applehelp_codesign_flags', . at 0xf5b50fa0>, 'applehelp') [app] adding config value: ('applehelp_indexer_path', '/usr/bin/hiutil', 'applehelp') [app] adding config value: ('applehelp_codesign_path', '/usr/bin/codesign', 'applehelp') [app] adding config value: ('applehelp_disable_external_tools', False, 'applehelp') [app] setting up extension: 'sphinxcontrib.devhelp' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding config value: ('devhelp_basename', . at 0xf5b5b028>, 'devhelp') [app] setting up extension: 'sphinxcontrib.htmlhelp' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding config value: ('htmlhelp_basename', , None) [app] adding config value: ('htmlhelp_file_suffix', None, 'html', []) [app] adding config value: ('htmlhelp_link_suffix', None, 'html', []) [app] setting up extension: 'sphinxcontrib.serializinghtml' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding builder: [app] setting up extension: 'sphinxcontrib.qthelp' [app] setting up extension: 'sphinx.builders.html' [app] adding builder: [app] adding config value: ('qthelp_basename', . at 0xf5c00a90>, 'html') [app] adding config value: ('qthelp_namespace', None, 'html', []) [app] adding config value: ('qthelp_theme', 'nonav', 'html') [app] adding config value: ('qthelp_theme_options', {}, 'html') [app] setting up extension: 'alabaster' [app] connecting event 'html-page-context': [id=52] [app] setting up extension: 'sphinx.ext.autodoc' [app] adding autodocumenter: [app] adding directive: ('automodule', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autoclass', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autoexception', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autodata', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autodatadecl', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autofunction', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autodecorator', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('automethod', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autoattribute', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autoproperty', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autoinstanceattribute', , None, None, {}) [app] adding autodocumenter: [app] adding directive: ('autoslotsattribute', , None, None, {}) [app] adding config value: ('autoclass_content', 'class', True) [app] adding config value: ('autodoc_member_order', 'alphabetic', True) [app] adding config value: ('autodoc_default_flags', [], True) [app] adding config value: ('autodoc_default_options', {}, True) [app] adding config value: ('autodoc_docstring_signature', True, True) [app] adding config value: ('autodoc_mock_imports', [], True) [app] adding config value: ('autodoc_typehints', 'signature', True, ) [app] adding config value: ('autodoc_warningiserror', True, True) [app] adding config value: ('autodoc_inherit_docstrings', True, True) [app] adding event: 'autodoc-before-process-signature' [app] adding event: 'autodoc-process-docstring' [app] adding event: 'autodoc-process-signature' [app] adding event: 'autodoc-skip-member' [app] connecting event 'config-inited': [id=53] [app] setting up extension: 'sphinx.ext.autodoc.type_comment' [app] connecting event 'autodoc-before-process-signature': [id=54] [app] setting up extension: 'sphinx.ext.doctest' [app] adding directive: ('testsetup', , None, None, {}) [app] adding directive: ('testcleanup', , None, None, {}) [app] adding directive: ('doctest', , None, None, {}) [app] adding directive: ('testcode', , None, None, {}) [app] adding directive: ('testoutput', , None, None, {}) [app] adding builder: [app] adding config value: ('doctest_path', [], False) [app] adding config value: ('doctest_test_doctest_blocks', 'default', False) [app] adding config value: ('doctest_global_setup', '', False) [app] adding config value: ('doctest_global_cleanup', '', False) [app] adding config value: ('doctest_default_flags', 41, False) [app] setting up extension: 'sphinx.ext.intersphinx' [app] adding config value: ('intersphinx_mapping', {}, True) [app] adding config value: ('intersphinx_cache_limit', 5, False) [app] adding config value: ('intersphinx_timeout', None, False) [app] connecting event 'config-inited': [id=55] [app] connecting event 'builder-inited': [id=56] [app] connecting event 'missing-reference': [id=57] [app] setting up extension: 'sphinx.ext.todo' [app] adding event: 'todo-defined' [app] adding config value: ('todo_include_todos', False, 'html') [app] adding config value: ('todo_link_only', False, 'html') [app] adding config value: ('todo_emit_warnings', False, 'html') [app] adding node: (, {}) [app] adding translation_handlers: , {} [app] adding node: (, {'html': (, ), 'latex': (, ), 'text': (, ), 'man': (, ), 'texinfo': (, )}) [app] adding translation_handlers: , {'html': (, ), 'latex': (, ), 'text': (, ), 'man': (, ), 'texinfo': (, )} [app] adding directive: ('todo', , None, None, {}) [app] adding directive: ('todolist', , None, None, {}) [app] adding domain: [app] connecting event 'doctree-resolved': [id=58] [app] setting up extension: 'sphinx.ext.coverage' [app] adding builder: [app] adding config value: ('coverage_ignore_modules', [], False) [app] adding config value: ('coverage_ignore_functions', [], False) [app] adding config value: ('coverage_ignore_classes', [], False) [app] adding config value: ('coverage_ignore_pyobjects', [], False) [app] adding config value: ('coverage_c_path', [], False) [app] adding config value: ('coverage_c_regexes', {}, False) [app] adding config value: ('coverage_ignore_c_items', {}, False) [app] adding config value: ('coverage_write_headline', True, False) [app] adding config value: ('coverage_skip_undoc_in_source', False, False) [app] setting up extension: 'numpydoc' [app] connecting event 'autodoc-process-docstring': [id=59] [app] connecting event 'autodoc-process-signature': [id=60] [app] adding config value: ('numpydoc_edit_link', None, False) [app] adding config value: ('numpydoc_use_plots', None, False) [app] adding config value: ('numpydoc_show_class_members', True, True) [app] adding config value: ('numpydoc_show_inherited_class_members', True, True) [app] adding config value: ('numpydoc_class_members_toctree', True, True) [app] adding config value: ('numpydoc_citation_re', '[a-z0-9_.-]+', True) [app] adding domain: [app] adding domain: making output directory... done [app] emitting event: 'config-inited'(,) WARNING: html_static_path entry '_static' does not exist loading pickled environment... done [app] emitting event: 'builder-inited'() loading intersphinx inventory from http://docs.python.org/objects.inv... WARNING: failed to reach any of the inventories with the following issues: intersphinx inventory 'http://docs.python.org/objects.inv' not fetchable due to : HTTPConnectionPool(host='docs.python.org', port=80): Max retries exceeded with url: /objects.inv (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')) building [mo]: targets for 0 po files that are out of date building [html]: targets for 5 source files that are out of date updating environment: [app] emitting event: 'env-get-outdated'(, set(), set(), set()) 0 added, 0 changed, 0 removed [app] emitting event: 'env-before-read-docs'(, []) [app] emitting event: 'env-updated'(,) looking for now-outdated files... [app] emitting event: 'env-get-updated'(,) none found docnames to write: index, install, intro, reference, tutorial preparing documents... done writing output... [ 20%] index [app] emitting event: 'doctree-resolved'(
, 'index') SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for python-blosc documentation master file, created by sphinx-quickstart on Sun May 5 19:38:24 2013. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive.
Welcome to python-blosc’s documentation!Contents:IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities
Indices and tablesIndexModule IndexSearch Page
SphinxTranslator.dispatch_visit calling visit_comment for python-blosc documentation master file, created by sphinx-quickstart on Sun May 5 19:38:24 2013. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. SphinxTranslator.dispatch_visit calling visit_section for
Welcome to python-blosc’s documentation!Contents:IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities
SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_paragraph for Contents: SphinxTranslator.dispatch_visit calling visit_Text for Contents: SphinxTranslator.dispatch_departure calling depart_Text for Contents: SphinxTranslator.dispatch_departure calling depart_paragraph for Contents: SphinxTranslator.dispatch_visit calling visit_compound for IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_compact_paragraph for IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_bullet_list for IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_list_item for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Introduction SphinxTranslator.dispatch_visit calling visit_reference for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_reference for Introduction SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Introduction SphinxTranslator.dispatch_visit calling visit_bullet_list for What is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_list_item for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_compact_paragraph for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_reference for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_Text for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_Text for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_reference for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_compact_paragraph for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_list_item for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_list_item for Mailing list SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Mailing list SphinxTranslator.dispatch_visit calling visit_reference for Mailing list SphinxTranslator.dispatch_visit calling visit_Text for Mailing list SphinxTranslator.dispatch_departure calling depart_Text for Mailing list SphinxTranslator.dispatch_departure calling depart_reference for Mailing list SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Mailing list SphinxTranslator.dispatch_departure calling depart_list_item for Mailing list SphinxTranslator.dispatch_departure calling depart_bullet_list for What is python-blosc?Mailing list SphinxTranslator.dispatch_departure calling depart_list_item for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_list_item for InstallationInstalling from PyPI repositoryBuilding manuallyTestingInstalling SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Installation SphinxTranslator.dispatch_visit calling visit_reference for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_reference for Installation SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Installation SphinxTranslator.dispatch_visit calling visit_bullet_list for Installing from PyPI repositoryBuilding manuallyTestingInstalling SphinxTranslator.dispatch_visit calling visit_list_item for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_reference for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_Text for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_Text for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_reference for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_list_item for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_list_item for Building manually SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Building manually SphinxTranslator.dispatch_visit calling visit_reference for Building manually SphinxTranslator.dispatch_visit calling visit_Text for Building manually SphinxTranslator.dispatch_departure calling depart_Text for Building manually SphinxTranslator.dispatch_departure calling depart_reference for Building manually SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Building manually SphinxTranslator.dispatch_departure calling depart_list_item for Building manually SphinxTranslator.dispatch_visit calling visit_list_item for Testing SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Testing SphinxTranslator.dispatch_visit calling visit_reference for Testing SphinxTranslator.dispatch_visit calling visit_Text for Testing SphinxTranslator.dispatch_departure calling depart_Text for Testing SphinxTranslator.dispatch_departure calling depart_reference for Testing SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Testing SphinxTranslator.dispatch_departure calling depart_list_item for Testing SphinxTranslator.dispatch_visit calling visit_list_item for Installing SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Installing SphinxTranslator.dispatch_visit calling visit_reference for Installing SphinxTranslator.dispatch_visit calling visit_Text for Installing SphinxTranslator.dispatch_departure calling depart_Text for Installing SphinxTranslator.dispatch_departure calling depart_reference for Installing SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Installing SphinxTranslator.dispatch_departure calling depart_list_item for Installing SphinxTranslator.dispatch_departure calling depart_bullet_list for Installing from PyPI repositoryBuilding manuallyTestingInstalling SphinxTranslator.dispatch_departure calling depart_list_item for InstallationInstalling from PyPI repositoryBuilding manuallyTestingInstalling SphinxTranslator.dispatch_visit calling visit_list_item for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Tutorials SphinxTranslator.dispatch_visit calling visit_reference for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_reference for Tutorials SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Tutorials SphinxTranslator.dispatch_visit calling visit_bullet_list for Compressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_list_item for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_reference for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_Text for Compressing and decompressing with SphinxTranslator.dispatch_departure calling depart_Text for Compressing and decompressing with SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_departure calling depart_reference for Compressing and decompressing with blosc SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Compressing and decompressing with blosc SphinxTranslator.dispatch_departure calling depart_list_item for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_list_item for Using different filters SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Using different filters SphinxTranslator.dispatch_visit calling visit_reference for Using different filters SphinxTranslator.dispatch_visit calling visit_Text for Using different filters SphinxTranslator.dispatch_departure calling depart_Text for Using different filters SphinxTranslator.dispatch_departure calling depart_reference for Using different filters SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Using different filters SphinxTranslator.dispatch_departure calling depart_list_item for Using different filters SphinxTranslator.dispatch_visit calling visit_list_item for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_reference for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_Text for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_Text for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_reference for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_list_item for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_list_item for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_reference for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_Text for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_Text for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_reference for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_list_item for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_list_item for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_reference for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_Text for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_Text for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_reference for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_list_item for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_list_item for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_reference for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_Text for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_Text for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_reference for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_list_item for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_list_item for Links to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Links to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_reference for Links to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_Text for Links to external discussions on SphinxTranslator.dispatch_departure calling depart_Text for Links to external discussions on SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for optimization SphinxTranslator.dispatch_departure calling depart_Text for optimization SphinxTranslator.dispatch_departure calling depart_reference for Links to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Links to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_list_item for Links to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_bullet_list for Compressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_list_item for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_list_item for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Library Reference SphinxTranslator.dispatch_visit calling visit_reference for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_reference for Library Reference SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Library Reference SphinxTranslator.dispatch_visit calling visit_bullet_list for First level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_list_item for First level variables SphinxTranslator.dispatch_visit calling visit_compact_paragraph for First level variables SphinxTranslator.dispatch_visit calling visit_reference for First level variables SphinxTranslator.dispatch_visit calling visit_Text for First level variables SphinxTranslator.dispatch_departure calling depart_Text for First level variables SphinxTranslator.dispatch_departure calling depart_reference for First level variables SphinxTranslator.dispatch_departure calling depart_compact_paragraph for First level variables SphinxTranslator.dispatch_departure calling depart_list_item for First level variables SphinxTranslator.dispatch_visit calling visit_list_item for Public functions SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Public functions SphinxTranslator.dispatch_visit calling visit_reference for Public functions SphinxTranslator.dispatch_visit calling visit_Text for Public functions SphinxTranslator.dispatch_departure calling depart_Text for Public functions SphinxTranslator.dispatch_departure calling depart_reference for Public functions SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Public functions SphinxTranslator.dispatch_departure calling depart_list_item for Public functions SphinxTranslator.dispatch_visit calling visit_list_item for Utilities SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Utilities SphinxTranslator.dispatch_visit calling visit_reference for Utilities SphinxTranslator.dispatch_visit calling visit_Text for Utilities SphinxTranslator.dispatch_departure calling depart_Text for Utilities SphinxTranslator.dispatch_departure calling depart_reference for Utilities SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Utilities SphinxTranslator.dispatch_departure calling depart_list_item for Utilities SphinxTranslator.dispatch_departure calling depart_bullet_list for First level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_list_item for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_bullet_list for IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_compact_paragraph for IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_compound for IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_section for
Welcome to python-blosc’s documentation!Contents:IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities
SphinxTranslator.dispatch_visit calling visit_section for
Indices and tablesIndexModule IndexSearch Page
SphinxTranslator.dispatch_visit calling visit_title for Indices and tables SphinxTranslator.dispatch_visit calling visit_Text for Indices and tables SphinxTranslator.dispatch_departure calling depart_Text for Indices and tables SphinxTranslator.dispatch_departure calling depart_title for Indices and tables SphinxTranslator.dispatch_visit calling visit_bullet_list for IndexModule IndexSearch Page SphinxTranslator.dispatch_visit calling visit_list_item for Index SphinxTranslator.dispatch_visit calling visit_paragraph for Index SphinxTranslator.dispatch_visit calling visit_reference for Index SphinxTranslator.dispatch_visit calling visit_inline for Index SphinxTranslator.dispatch_visit calling visit_Text for Index SphinxTranslator.dispatch_departure calling depart_Text for Index SphinxTranslator.dispatch_departure calling depart_inline for Index SphinxTranslator.dispatch_departure calling depart_reference for Index SphinxTranslator.dispatch_departure calling depart_paragraph for Index SphinxTranslator.dispatch_departure calling depart_list_item for Index SphinxTranslator.dispatch_visit calling visit_list_item for Module Index SphinxTranslator.dispatch_visit calling visit_paragraph for Module Index SphinxTranslator.dispatch_visit calling visit_reference for Module Index SphinxTranslator.dispatch_visit calling visit_inline for Module Index SphinxTranslator.dispatch_visit calling visit_Text for Module Index SphinxTranslator.dispatch_departure calling depart_Text for Module Index SphinxTranslator.dispatch_departure calling depart_inline for Module Index SphinxTranslator.dispatch_departure calling depart_reference for Module Index SphinxTranslator.dispatch_departure calling depart_paragraph for Module Index SphinxTranslator.dispatch_departure calling depart_list_item for Module Index SphinxTranslator.dispatch_visit calling visit_list_item for Search Page SphinxTranslator.dispatch_visit calling visit_paragraph for Search Page SphinxTranslator.dispatch_visit calling visit_reference for Search Page SphinxTranslator.dispatch_visit calling visit_inline for Search Page SphinxTranslator.dispatch_visit calling visit_Text for Search Page SphinxTranslator.dispatch_departure calling depart_Text for Search Page SphinxTranslator.dispatch_departure calling depart_inline for Search Page SphinxTranslator.dispatch_departure calling depart_reference for Search Page SphinxTranslator.dispatch_departure calling depart_paragraph for Search Page SphinxTranslator.dispatch_departure calling depart_list_item for Search Page SphinxTranslator.dispatch_departure calling depart_bullet_list for IndexModule IndexSearch Page SphinxTranslator.dispatch_departure calling depart_section for
Indices and tablesIndexModule IndexSearch Page
SphinxTranslator.dispatch_departure calling depart_document for python-blosc documentation master file, created by sphinx-quickstart on Sun May 5 19:38:24 2013. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive.
Welcome to python-blosc’s documentation!Contents:IntroductionWhat is python-blosc?Mailing listInstallationInstalling from PyPI repositoryBuilding manuallyTestingInstallingTutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimizationLibrary ReferenceFirst level variablesPublic functionsUtilities
Indices and tablesIndexModule IndexSearch Page
SphinxTranslator.dispatch_visit calling visit_document for Introduction SphinxTranslator.dispatch_visit calling visit_title for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_title for Introduction SphinxTranslator.dispatch_departure calling depart_document for Introduction SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation!Indices and tables SphinxTranslator.dispatch_visit calling visit_bullet_list for Welcome to python-blosc’s documentation!Indices and tables SphinxTranslator.dispatch_visit calling visit_list_item for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_reference for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_reference for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_bullet_list for SphinxTranslator.dispatch_departure calling depart_list_item for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_list_item for Indices and tables SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Indices and tables SphinxTranslator.dispatch_visit calling visit_reference for Indices and tables SphinxTranslator.dispatch_visit calling visit_Text for Indices and tables SphinxTranslator.dispatch_departure calling depart_Text for Indices and tables SphinxTranslator.dispatch_departure calling depart_reference for Indices and tables SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Indices and tables SphinxTranslator.dispatch_departure calling depart_list_item for Indices and tables SphinxTranslator.dispatch_departure calling depart_bullet_list for Welcome to python-blosc’s documentation!Indices and tables SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation!Indices and tables [app] emitting event: 'html-page-context'('index', 'page.html', {'embedded': False, 'project': 'python-blosc', 'release': '1.5.0', 'version': writing output... [ 40%] install [app] emitting event: 'doctree-resolved'(>, 'install') SphinxTranslator.dispatch_visit calling visit_document for Installation SphinxTranslator.dispatch_visit calling visit_title for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_title for Installation SphinxTranslator.dispatch_departure calling depart_document for Installation SphinxTranslator.dispatch_visit calling visit_document for
Installationpython-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested).Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too.This package supports Python 2.6, 2.7 and 3.3 or higher versions.
Installing from PyPI repositoryDo:$ pip install -U blosc
Building manuallyFirst, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources.Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not.
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
TestingAfter compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py:$ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode)Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite:$ nosetests --with-doctest (add -v for verbose mode)Once installed, you can re-run the tests at any time with:$ python -c "import blosc; blosc.test()"
InstallingInstall it as a typical Python package:$ python setup.py install
SphinxTranslator.dispatch_visit calling visit_section for
Installationpython-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested).Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too.This package supports Python 2.6, 2.7 and 3.3 or higher versions.
Installing from PyPI repositoryDo:$ pip install -U blosc
Building manuallyFirst, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources.Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not.
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
TestingAfter compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py:$ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode)Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite:$ nosetests --with-doctest (add -v for verbose mode)Once installed, you can re-run the tests at any time with:$ python -c "import blosc; blosc.test()"
InstallingInstall it as a typical Python package:$ python setup.py install
SphinxTranslator.dispatch_visit calling visit_title for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_title for Installation SphinxTranslator.dispatch_visit calling visit_paragraph for python-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested). SphinxTranslator.dispatch_visit calling visit_Text for python-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested). SphinxTranslator.dispatch_departure calling depart_Text for python-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested). SphinxTranslator.dispatch_departure calling depart_paragraph for python-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested). SphinxTranslator.dispatch_visit calling visit_paragraph for Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too. SphinxTranslator.dispatch_visit calling visit_Text for Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too. SphinxTranslator.dispatch_departure calling depart_Text for Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too. SphinxTranslator.dispatch_departure calling depart_paragraph for Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too. SphinxTranslator.dispatch_visit calling visit_paragraph for This package supports Python 2.6, 2.7 and 3.3 or higher versions. SphinxTranslator.dispatch_visit calling visit_Text for This package supports Python 2.6, 2.7 and 3.3 or higher versions. SphinxTranslator.dispatch_departure calling depart_Text for This package supports Python 2.6, 2.7 and 3.3 or higher versions. SphinxTranslator.dispatch_departure calling depart_paragraph for This package supports Python 2.6, 2.7 and 3.3 or higher versions. SphinxTranslator.dispatch_visit calling visit_section for
Installing from PyPI repositoryDo:$ pip install -U blosc
SphinxTranslator.dispatch_visit calling visit_title for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_Text for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_Text for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_title for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_paragraph for Do: SphinxTranslator.dispatch_visit calling visit_Text for Do: SphinxTranslator.dispatch_departure calling depart_Text for Do: SphinxTranslator.dispatch_departure calling depart_paragraph for Do: SphinxTranslator.dispatch_visit calling visit_literal_block for $ pip install -U blosc SphinxTranslator.dispatch_departure calling depart_section for
Installing from PyPI repositoryDo:$ pip install -U blosc
SphinxTranslator.dispatch_visit calling visit_section for
Building manuallyFirst, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources.Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not.
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
SphinxTranslator.dispatch_visit calling visit_title for Building manually SphinxTranslator.dispatch_visit calling visit_Text for Building manually SphinxTranslator.dispatch_departure calling depart_Text for Building manually SphinxTranslator.dispatch_departure calling depart_title for Building manually SphinxTranslator.dispatch_visit calling visit_paragraph for First, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources. SphinxTranslator.dispatch_visit calling visit_Text for First, go to the python-blosc official repository at SphinxTranslator.dispatch_departure calling depart_Text for First, go to the python-blosc official repository at SphinxTranslator.dispatch_visit calling visit_reference for https://github.com/Blosc/python-blosc SphinxTranslator.dispatch_visit calling visit_Text for https://github.com/Blosc/python-blosc SphinxTranslator.dispatch_departure calling depart_Text for https://github.com/Blosc/python-blosc SphinxTranslator.dispatch_departure calling depart_reference for https://github.com/Blosc/python-blosc SphinxTranslator.dispatch_visit calling visit_Text for and download the sources. SphinxTranslator.dispatch_departure calling depart_Text for and download the sources. SphinxTranslator.dispatch_departure calling depart_paragraph for First, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources. SphinxTranslator.dispatch_visit calling visit_paragraph for Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not. SphinxTranslator.dispatch_visit calling visit_Text for Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not. SphinxTranslator.dispatch_departure calling depart_Text for Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not. SphinxTranslator.dispatch_departure calling depart_paragraph for Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not. SphinxTranslator.dispatch_visit calling visit_section for
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
SphinxTranslator.dispatch_visit calling visit_title for Compiling without an installed Blosc library SphinxTranslator.dispatch_visit calling visit_Text for Compiling without an installed Blosc library SphinxTranslator.dispatch_departure calling depart_Text for Compiling without an installed Blosc library SphinxTranslator.dispatch_departure calling depart_title for Compiling without an installed Blosc library SphinxTranslator.dispatch_visit calling visit_paragraph for python-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do: SphinxTranslator.dispatch_visit calling visit_Text for python-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do: SphinxTranslator.dispatch_departure calling depart_Text for python-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do: SphinxTranslator.dispatch_departure calling depart_paragraph for python-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do: SphinxTranslator.dispatch_visit calling visit_literal_block for $ python setup.py build_ext --inplace SphinxTranslator.dispatch_visit calling visit_paragraph for That’s all. You can proceed with testing section now. SphinxTranslator.dispatch_visit calling visit_Text for That’s all. You can proceed with testing section now. SphinxTranslator.dispatch_departure calling depart_Text for That’s all. You can proceed with testing section now. SphinxTranslator.dispatch_departure calling depart_paragraph for That’s all. You can proceed with testing section now. SphinxTranslator.dispatch_departure calling depart_section for
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
SphinxTranslator.dispatch_visit calling visit_section for
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
SphinxTranslator.dispatch_visit calling visit_title for Compiling with an installed Blosc library SphinxTranslator.dispatch_visit calling visit_Text for Compiling with an installed Blosc library SphinxTranslator.dispatch_departure calling depart_Text for Compiling with an installed Blosc library SphinxTranslator.dispatch_departure calling depart_title for Compiling with an installed Blosc library SphinxTranslator.dispatch_visit calling visit_paragraph for In case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways. SphinxTranslator.dispatch_visit calling visit_Text for In case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways. SphinxTranslator.dispatch_departure calling depart_Text for In case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways. SphinxTranslator.dispatch_departure calling depart_paragraph for In case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways. SphinxTranslator.dispatch_visit calling visit_paragraph for Using an environment variable: SphinxTranslator.dispatch_visit calling visit_Text for Using an environment variable: SphinxTranslator.dispatch_departure calling depart_Text for Using an environment variable: SphinxTranslator.dispatch_departure calling depart_paragraph for Using an environment variable: SphinxTranslator.dispatch_visit calling visit_literal_block for $ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplace SphinxTranslator.dispatch_visit calling visit_paragraph for Using a flag: SphinxTranslator.dispatch_visit calling visit_Text for Using a flag: SphinxTranslator.dispatch_departure calling depart_Text for Using a flag: SphinxTranslator.dispatch_departure calling depart_paragraph for Using a flag: SphinxTranslator.dispatch_visit calling visit_literal_block for $ python setup.py build_ext --inplace --blosc=/usr/local SphinxTranslator.dispatch_departure calling depart_section for
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
SphinxTranslator.dispatch_visit calling visit_section for
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
SphinxTranslator.dispatch_visit calling visit_title for Generating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_Text for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_Text for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_title for Generating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_paragraph for In case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do: SphinxTranslator.dispatch_visit calling visit_Text for In case you want to generate the documentation locally, you will need to have the SphinxTranslator.dispatch_departure calling depart_Text for In case you want to generate the documentation locally, you will need to have the SphinxTranslator.dispatch_visit calling visit_literal for Sphinx SphinxTranslator.dispatch_visit calling visit_Text for Sphinx SphinxTranslator.dispatch_departure calling depart_Text for Sphinx SphinxTranslator.dispatch_departure calling depart_literal for Sphinx SphinxTranslator.dispatch_visit calling visit_Text for documentation system, as well as the SphinxTranslator.dispatch_departure calling depart_Text for documentation system, as well as the SphinxTranslator.dispatch_visit calling visit_literal for numpydoc SphinxTranslator.dispatch_visit calling visit_Text for numpydoc SphinxTranslator.dispatch_departure calling depart_Text for numpydoc SphinxTranslator.dispatch_departure calling depart_literal for numpydoc SphinxTranslator.dispatch_visit calling visit_Text for extension, installed. Then go down to SphinxTranslator.dispatch_departure calling depart_Text for extension, installed. Then go down to SphinxTranslator.dispatch_visit calling visit_literal for doc/ SphinxTranslator.dispatch_visit calling visit_Text for doc/ SphinxTranslator.dispatch_departure calling depart_Text for doc/ SphinxTranslator.dispatch_departure calling depart_literal for doc/ SphinxTranslator.dispatch_visit calling visit_Text for directory and do: SphinxTranslator.dispatch_departure calling depart_Text for directory and do: SphinxTranslator.dispatch_departure calling depart_paragraph for In case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do: SphinxTranslator.dispatch_visit calling visit_literal_block for $ make html|latex|latexpdf SphinxTranslator.dispatch_departure calling depart_section for
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
SphinxTranslator.dispatch_departure calling depart_section for
Building manuallyFirst, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources.Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not.
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
SphinxTranslator.dispatch_visit calling visit_section for
TestingAfter compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py:$ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode)Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite:$ nosetests --with-doctest (add -v for verbose mode)Once installed, you can re-run the tests at any time with:$ python -c "import blosc; blosc.test()"
SphinxTranslator.dispatch_visit calling visit_title for Testing SphinxTranslator.dispatch_visit calling visit_Text for Testing SphinxTranslator.dispatch_departure calling depart_Text for Testing SphinxTranslator.dispatch_departure calling depart_title for Testing SphinxTranslator.dispatch_visit calling visit_paragraph for After compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py: SphinxTranslator.dispatch_visit calling visit_Text for After compiling, you can quickly check that the package is sane by running the doctests in SphinxTranslator.dispatch_departure calling depart_Text for After compiling, you can quickly check that the package is sane by running the doctests in SphinxTranslator.dispatch_visit calling visit_literal for blosc/test.py SphinxTranslator.dispatch_visit calling visit_Text for blosc/test.py SphinxTranslator.dispatch_departure calling depart_Text for blosc/test.py SphinxTranslator.dispatch_departure calling depart_literal for blosc/test.py SphinxTranslator.dispatch_visit calling visit_Text for : SphinxTranslator.dispatch_departure calling depart_Text for : SphinxTranslator.dispatch_departure calling depart_paragraph for After compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py: SphinxTranslator.dispatch_visit calling visit_literal_block for $ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode) SphinxTranslator.dispatch_visit calling visit_paragraph for Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite: SphinxTranslator.dispatch_visit calling visit_Text for Or alternatively, you can use the third-party SphinxTranslator.dispatch_departure calling depart_Text for Or alternatively, you can use the third-party SphinxTranslator.dispatch_visit calling visit_literal for nosetests SphinxTranslator.dispatch_visit calling visit_Text for nosetests SphinxTranslator.dispatch_departure calling depart_Text for nosetests SphinxTranslator.dispatch_departure calling depart_literal for nosetests SphinxTranslator.dispatch_visit calling visit_Text for script to run both the doctests and the test suite: SphinxTranslator.dispatch_departure calling depart_Text for script to run both the doctests and the test suite: SphinxTranslator.dispatch_departure calling depart_paragraph for Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite: SphinxTranslator.dispatch_visit calling visit_literal_block for $ nosetests --with-doctest (add -v for verbose mode) SphinxTranslator.dispatch_visit calling visit_paragraph for Once installed, you can re-run the tests at any time with: SphinxTranslator.dispatch_visit calling visit_Text for Once installed, you can re-run the tests at any time with: SphinxTranslator.dispatch_departure calling depart_Text for Once installed, you can re-run the tests at any time with: SphinxTranslator.dispatch_departure calling depart_paragraph for Once installed, you can re-run the tests at any time with: SphinxTranslator.dispatch_visit calling visit_literal_block for $ python -c "import blosc; blosc.test()" SphinxTranslator.dispatch_departure calling depart_section for
TestingAfter compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py:$ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode)Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite:$ nosetests --with-doctest (add -v for verbose mode)Once installed, you can re-run the tests at any time with:$ python -c "import blosc; blosc.test()"
SphinxTranslator.dispatch_visit calling visit_section for
InstallingInstall it as a typical Python package:$ python setup.py install
SphinxTranslator.dispatch_visit calling visit_title for Installing SphinxTranslator.dispatch_visit calling visit_Text for Installing SphinxTranslator.dispatch_departure calling depart_Text for Installing SphinxTranslator.dispatch_departure calling depart_title for Installing SphinxTranslator.dispatch_visit calling visit_paragraph for Install it as a typical Python package: SphinxTranslator.dispatch_visit calling visit_Text for Install it as a typical Python package: SphinxTranslator.dispatch_departure calling depart_Text for Install it as a typical Python package: SphinxTranslator.dispatch_departure calling depart_paragraph for Install it as a typical Python package: SphinxTranslator.dispatch_visit calling visit_literal_block for $ python setup.py install SphinxTranslator.dispatch_departure calling depart_section for
InstallingInstall it as a typical Python package:$ python setup.py install
SphinxTranslator.dispatch_departure calling depart_section for
Installationpython-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested).Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too.This package supports Python 2.6, 2.7 and 3.3 or higher versions.
Installing from PyPI repositoryDo:$ pip install -U blosc
Building manuallyFirst, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources.Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not.
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
TestingAfter compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py:$ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode)Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite:$ nosetests --with-doctest (add -v for verbose mode)Once installed, you can re-run the tests at any time with:$ python -c "import blosc; blosc.test()"
InstallingInstall it as a typical Python package:$ python setup.py install
SphinxTranslator.dispatch_departure calling depart_document for
Installationpython-blosc comes with C Blosc sources, so it does not depend on any other library (bar Python itself of course). Of course, if you are going to install from sources, you are going to need a C compiler (GCC, clang and MSVC 2008/2010/2012/2015 have been tested).Also, there are situations where you may want to link with an already existing Blosc library in your system. You can do that too.This package supports Python 2.6, 2.7 and 3.3 or higher versions.
Installing from PyPI repositoryDo:$ pip install -U blosc
Building manuallyFirst, go to the python-blosc official repository at https://github.com/Blosc/python-blosc and download the sources.Then, there are different ways to compile python-blosc, depending on whether you want to link with an already installed Blosc library or not.
Compiling without an installed Blosc librarypython-blosc comes with the Blosc sources with it so, assuming that you have a C compiler installed, do:$ python setup.py build_ext --inplaceThat’s all. You can proceed with testing section now.
Compiling with an installed Blosc libraryIn case you have Blosc installed as an external library (and disregard the included Blosc sources) you can link with it in a couple of ways.Using an environment variable:$ BLOSC_DIR=/usr/local (or "set BLOSC_DIR=\blosc" on Win) $ export BLOSC_DIR (not needed on Win) $ python setup.py build_ext --inplaceUsing a flag:$ python setup.py build_ext --inplace --blosc=/usr/local
Generating Sphinx documentationIn case you want to generate the documentation locally, you will need to have the Sphinx documentation system, as well as the numpydoc extension, installed. Then go down to doc/ directory and do:$ make html|latex|latexpdf
TestingAfter compiling, you can quickly check that the package is sane by running the doctests in blosc/test.py:$ PYTHONPATH=. (or "set PYTHONPATH=." on Win) $ export PYTHONPATH=. (not needed on Win) $ python blosc/test.py (add -v for verbose mode)Or alternatively, you can use the third-party nosetests script to run both the doctests and the test suite:$ nosetests --with-doctest (add -v for verbose mode)Once installed, you can re-run the tests at any time with:$ python -c "import blosc; blosc.test()"
InstallingInstall it as a typical Python package:$ python setup.py install
SphinxTranslator.dispatch_visit calling visit_document for Tutorials SphinxTranslator.dispatch_visit calling visit_title for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_title for Tutorials SphinxTranslator.dispatch_departure calling depart_document for Tutorials SphinxTranslator.dispatch_visit calling visit_document for Introduction SphinxTranslator.dispatch_visit calling visit_title for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_title for Introduction SphinxTranslator.dispatch_departure calling depart_document for Introduction SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for Installation SphinxTranslator.dispatch_visit calling visit_title for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_title for Installation SphinxTranslator.dispatch_departure calling depart_document for Installation SphinxTranslator.dispatch_visit calling visit_document for InstallationInstalling from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_visit calling visit_bullet_list for InstallationInstalling from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_visit calling visit_list_item for InstallationInstalling from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Installation SphinxTranslator.dispatch_visit calling visit_reference for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_reference for Installation SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Installation SphinxTranslator.dispatch_visit calling visit_bullet_list for Installing from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_visit calling visit_list_item for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_reference for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_Text for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_Text for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_reference for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Installing from PyPI repository SphinxTranslator.dispatch_departure calling depart_list_item for Installing from PyPI repository SphinxTranslator.dispatch_visit calling visit_list_item for Building manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Building manually SphinxTranslator.dispatch_visit calling visit_reference for Building manually SphinxTranslator.dispatch_visit calling visit_Text for Building manually SphinxTranslator.dispatch_departure calling depart_Text for Building manually SphinxTranslator.dispatch_departure calling depart_reference for Building manually SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Building manually SphinxTranslator.dispatch_visit calling visit_bullet_list for Compiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_list_item for Compiling without an installed Blosc library SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Compiling without an installed Blosc library SphinxTranslator.dispatch_visit calling visit_reference for Compiling without an installed Blosc library SphinxTranslator.dispatch_visit calling visit_Text for Compiling without an installed Blosc library SphinxTranslator.dispatch_departure calling depart_Text for Compiling without an installed Blosc library SphinxTranslator.dispatch_departure calling depart_reference for Compiling without an installed Blosc library SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Compiling without an installed Blosc library SphinxTranslator.dispatch_departure calling depart_list_item for Compiling without an installed Blosc library SphinxTranslator.dispatch_visit calling visit_list_item for Compiling with an installed Blosc library SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Compiling with an installed Blosc library SphinxTranslator.dispatch_visit calling visit_reference for Compiling with an installed Blosc library SphinxTranslator.dispatch_visit calling visit_Text for Compiling with an installed Blosc library SphinxTranslator.dispatch_departure calling depart_Text for Compiling with an installed Blosc library SphinxTranslator.dispatch_departure calling depart_reference for Compiling with an installed Blosc library SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Compiling with an installed Blosc library SphinxTranslator.dispatch_departure calling depart_list_item for Compiling with an installed Blosc library SphinxTranslator.dispatch_visit calling visit_list_item for Generating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Generating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_reference for Generating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_Text for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_Text for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_reference for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_list_item for Generating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_bullet_list for Compiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentation SphinxTranslator.dispatch_departure calling depart_list_item for Building manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentation SphinxTranslator.dispatch_visit calling visit_list_item for Testing SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Testing SphinxTranslator.dispatch_visit calling visit_reference for Testing SphinxTranslator.dispatch_visit calling visit_Text for Testing SphinxTranslator.dispatch_departure calling depart_Text for Testing SphinxTranslator.dispatch_departure calling depart_reference for Testing SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Testing SphinxTranslator.dispatch_departure calling depart_list_item for Testing SphinxTranslator.dispatch_visit calling visit_list_item for Installing SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Installing SphinxTranslator.dispatch_visit calling visit_reference for Installing SphinxTranslator.dispatch_visit calling visit_Text for Installing SphinxTranslator.dispatch_departure calling depart_Text for Installing SphinxTranslator.dispatch_departure calling depart_reference for Installing SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Installing SphinxTranslator.dispatch_departure calling depart_list_item for Installing SphinxTranslator.dispatch_departure calling depart_bullet_list for Installing from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_departure calling depart_list_item for InstallationInstalling from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_departure calling depart_bullet_list for InstallationInstalling from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling SphinxTranslator.dispatch_departure calling depart_document for InstallationInstalling from PyPI repositoryBuilding manuallyCompiling without an installed Blosc libraryCompiling with an installed Blosc libraryGenerating Sphinx documentationTestingInstalling [app] emitting event: 'html-page-context'('install', 'page.html', {'embedded': False, 'project': 'python-blosc', 'release': '1.5.0', 'version writing output... [ 60%] intro [app] emitting event: 'doctree-resolved'(>, 'intro') SphinxTranslator.dispatch_visit calling visit_document for Introduction SphinxTranslator.dispatch_visit calling visit_title for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_title for Introduction SphinxTranslator.dispatch_departure calling depart_document for Introduction SphinxTranslator.dispatch_visit calling visit_document for
Introduction
What is python-blosc?Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call.Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc.This is a Python package that wraps it.
Mailing listDiscussion about this module is welcome in the Blosc list:blosc@googlegroups.comhttp://groups.google.es/group/blosc
SphinxTranslator.dispatch_visit calling visit_section for
Introduction
What is python-blosc?Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call.Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc.This is a Python package that wraps it.
Mailing listDiscussion about this module is welcome in the Blosc list:blosc@googlegroups.comhttp://groups.google.es/group/blosc
SphinxTranslator.dispatch_visit calling visit_title for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_title for Introduction SphinxTranslator.dispatch_visit calling visit_section for
What is python-blosc?Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call.Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc.This is a Python package that wraps it.
SphinxTranslator.dispatch_visit calling visit_title for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_Text for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_Text for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_title for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_paragraph for Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. SphinxTranslator.dispatch_visit calling visit_Text for Blosc ( SphinxTranslator.dispatch_departure calling depart_Text for Blosc ( SphinxTranslator.dispatch_visit calling visit_reference for http://blosc.org SphinxTranslator.dispatch_visit calling visit_Text for http://blosc.org SphinxTranslator.dispatch_departure calling depart_Text for http://blosc.org SphinxTranslator.dispatch_departure calling depart_reference for http://blosc.org SphinxTranslator.dispatch_visit calling visit_Text for ) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. SphinxTranslator.dispatch_departure calling depart_Text for ) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. SphinxTranslator.dispatch_departure calling depart_paragraph for Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. SphinxTranslator.dispatch_visit calling visit_paragraph for Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. SphinxTranslator.dispatch_visit calling visit_Text for Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. SphinxTranslator.dispatch_departure calling depart_Text for Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. SphinxTranslator.dispatch_departure calling depart_paragraph for Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. SphinxTranslator.dispatch_visit calling visit_paragraph for This is a Python package that wraps it. SphinxTranslator.dispatch_visit calling visit_Text for This is a Python package that wraps it. SphinxTranslator.dispatch_departure calling depart_Text for This is a Python package that wraps it. SphinxTranslator.dispatch_departure calling depart_paragraph for This is a Python package that wraps it. SphinxTranslator.dispatch_departure calling depart_section for
What is python-blosc?Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call.Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc.This is a Python package that wraps it.
SphinxTranslator.dispatch_visit calling visit_section for
Mailing listDiscussion about this module is welcome in the Blosc list:blosc@googlegroups.comhttp://groups.google.es/group/blosc
SphinxTranslator.dispatch_visit calling visit_title for Mailing list SphinxTranslator.dispatch_visit calling visit_Text for Mailing list SphinxTranslator.dispatch_departure calling depart_Text for Mailing list SphinxTranslator.dispatch_departure calling depart_title for Mailing list SphinxTranslator.dispatch_visit calling visit_paragraph for Discussion about this module is welcome in the Blosc list: SphinxTranslator.dispatch_visit calling visit_Text for Discussion about this module is welcome in the Blosc list: SphinxTranslator.dispatch_departure calling depart_Text for Discussion about this module is welcome in the Blosc list: SphinxTranslator.dispatch_departure calling depart_paragraph for Discussion about this module is welcome in the Blosc list: SphinxTranslator.dispatch_visit calling visit_paragraph for blosc@googlegroups.com SphinxTranslator.dispatch_visit calling visit_reference for blosc@googlegroups.com SphinxTranslator.dispatch_visit calling visit_Text for blosc@googlegroups.com SphinxTranslator.dispatch_departure calling depart_Text for blosc@googlegroups.com SphinxTranslator.dispatch_departure calling depart_reference for blosc@googlegroups.com SphinxTranslator.dispatch_departure calling depart_paragraph for blosc@googlegroups.com SphinxTranslator.dispatch_visit calling visit_paragraph for http://groups.google.es/group/blosc SphinxTranslator.dispatch_visit calling visit_reference for http://groups.google.es/group/blosc SphinxTranslator.dispatch_visit calling visit_Text for http://groups.google.es/group/blosc SphinxTranslator.dispatch_departure calling depart_Text for http://groups.google.es/group/blosc SphinxTranslator.dispatch_departure calling depart_reference for http://groups.google.es/group/blosc SphinxTranslator.dispatch_departure calling depart_paragraph for http://groups.google.es/group/blosc SphinxTranslator.dispatch_departure calling depart_section for
Mailing listDiscussion about this module is welcome in the Blosc list:blosc@googlegroups.comhttp://groups.google.es/group/blosc
SphinxTranslator.dispatch_departure calling depart_section for
Introduction
What is python-blosc?Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call.Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc.This is a Python package that wraps it.
Mailing listDiscussion about this module is welcome in the Blosc list:blosc@googlegroups.comhttp://groups.google.es/group/blosc
SphinxTranslator.dispatch_departure calling depart_document for
Introduction
What is python-blosc?Blosc (http://blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call.Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc.This is a Python package that wraps it.
Mailing listDiscussion about this module is welcome in the Blosc list:blosc@googlegroups.comhttp://groups.google.es/group/blosc
SphinxTranslator.dispatch_visit calling visit_document for Installation SphinxTranslator.dispatch_visit calling visit_title for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_title for Installation SphinxTranslator.dispatch_departure calling depart_document for Installation SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for Introduction SphinxTranslator.dispatch_visit calling visit_title for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_title for Introduction SphinxTranslator.dispatch_departure calling depart_document for Introduction SphinxTranslator.dispatch_visit calling visit_document for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_bullet_list for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_list_item for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Introduction SphinxTranslator.dispatch_visit calling visit_reference for Introduction SphinxTranslator.dispatch_visit calling visit_Text for Introduction SphinxTranslator.dispatch_departure calling depart_Text for Introduction SphinxTranslator.dispatch_departure calling depart_reference for Introduction SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Introduction SphinxTranslator.dispatch_visit calling visit_bullet_list for What is python-blosc?Mailing list SphinxTranslator.dispatch_visit calling visit_list_item for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_compact_paragraph for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_reference for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_Text for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_Text for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_reference for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_compact_paragraph for What is python-blosc? SphinxTranslator.dispatch_departure calling depart_list_item for What is python-blosc? SphinxTranslator.dispatch_visit calling visit_list_item for Mailing list SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Mailing list SphinxTranslator.dispatch_visit calling visit_reference for Mailing list SphinxTranslator.dispatch_visit calling visit_Text for Mailing list SphinxTranslator.dispatch_departure calling depart_Text for Mailing list SphinxTranslator.dispatch_departure calling depart_reference for Mailing list SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Mailing list SphinxTranslator.dispatch_departure calling depart_list_item for Mailing list SphinxTranslator.dispatch_departure calling depart_bullet_list for What is python-blosc?Mailing list SphinxTranslator.dispatch_departure calling depart_list_item for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_departure calling depart_bullet_list for IntroductionWhat is python-blosc?Mailing list SphinxTranslator.dispatch_departure calling depart_document for IntroductionWhat is python-blosc?Mailing list [app] emitting event: 'html-page-context'('intro', 'page.html', {'embedded': False, 'project': 'python-blosc', 'release': '1.5.0', 'version': writing output... [ 80%] reference [app] emitting event: 'doctree-resolved'(>, 'reference') SphinxTranslator.dispatch_visit calling visit_document for Library Reference SphinxTranslator.dispatch_visit calling visit_title for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_title for Library Reference SphinxTranslator.dispatch_departure calling depart_document for Library Reference SphinxTranslator.dispatch_visit calling visit_document for
Library Reference
First level variables__version__The version of the blosc package.blosclib_versionThe version of the Blosc C library.clib_versionsA map for the versions of the compression libraries included in C library.cnamesThe list of compressors included in C library.cname2clibA map between compressor names and its libraries (or formats).ncoresThe number of cores detected.
Public functionsblosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) Trueblosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans Trueblosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray Trueblosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize Trueblosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize Trueblosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True
Utilitiesblosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version.blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names.blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system.blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>>blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library.blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512)blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True)blosc.print_versionsPrint all the versions of software that python-blosc relies on.
SphinxTranslator.dispatch_visit calling visit_section for
Library Reference
First level variables__version__The version of the blosc package.blosclib_versionThe version of the Blosc C library.clib_versionsA map for the versions of the compression libraries included in C library.cnamesThe list of compressors included in C library.cname2clibA map between compressor names and its libraries (or formats).ncoresThe number of cores detected.
Public functionsblosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) Trueblosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans Trueblosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray Trueblosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize Trueblosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize Trueblosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True
Utilitiesblosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version.blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names.blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system.blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>>blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library.blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512)blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True)blosc.print_versionsPrint all the versions of software that python-blosc relies on.
SphinxTranslator.dispatch_visit calling visit_title for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_title for Library Reference SphinxTranslator.dispatch_visit calling visit_section for
First level variables__version__The version of the blosc package.blosclib_versionThe version of the Blosc C library.clib_versionsA map for the versions of the compression libraries included in C library.cnamesThe list of compressors included in C library.cname2clibA map between compressor names and its libraries (or formats).ncoresThe number of cores detected.
SphinxTranslator.dispatch_visit calling visit_title for First level variables SphinxTranslator.dispatch_visit calling visit_Text for First level variables SphinxTranslator.dispatch_departure calling depart_Text for First level variables SphinxTranslator.dispatch_departure calling depart_title for First level variables SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for __version__The version of the blosc package. SphinxTranslator.dispatch_visit calling visit_desc_signature for __version__ SphinxTranslator.dispatch_visit calling visit_desc_name for __version__ SphinxTranslator.dispatch_visit calling visit_Text for __version__ SphinxTranslator.dispatch_departure calling depart_Text for __version__ SphinxTranslator.dispatch_departure calling depart_desc_name for __version__ SphinxTranslator.dispatch_departure calling depart_desc_signature for __version__ SphinxTranslator.dispatch_visit calling visit_desc_content for The version of the blosc package. SphinxTranslator.dispatch_visit calling visit_paragraph for The version of the blosc package. SphinxTranslator.dispatch_visit calling visit_Text for The version of the blosc package. SphinxTranslator.dispatch_departure calling depart_Text for The version of the blosc package. SphinxTranslator.dispatch_departure calling depart_paragraph for The version of the blosc package. SphinxTranslator.dispatch_departure calling depart_desc_content for The version of the blosc package. SphinxTranslator.dispatch_departure calling depart_desc for __version__The version of the blosc package. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosclib_versionThe version of the Blosc C library. SphinxTranslator.dispatch_visit calling visit_desc_signature for blosclib_version SphinxTranslator.dispatch_visit calling visit_desc_name for blosclib_version SphinxTranslator.dispatch_visit calling visit_Text for blosclib_version SphinxTranslator.dispatch_departure calling depart_Text for blosclib_version SphinxTranslator.dispatch_departure calling depart_desc_name for blosclib_version SphinxTranslator.dispatch_departure calling depart_desc_signature for blosclib_version SphinxTranslator.dispatch_visit calling visit_desc_content for The version of the Blosc C library. SphinxTranslator.dispatch_visit calling visit_paragraph for The version of the Blosc C library. SphinxTranslator.dispatch_visit calling visit_Text for The version of the Blosc C library. SphinxTranslator.dispatch_departure calling depart_Text for The version of the Blosc C library. SphinxTranslator.dispatch_departure calling depart_paragraph for The version of the Blosc C library. SphinxTranslator.dispatch_departure calling depart_desc_content for The version of the Blosc C library. SphinxTranslator.dispatch_departure calling depart_desc for blosclib_versionThe version of the Blosc C library. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for clib_versionsA map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_visit calling visit_desc_signature for clib_versions SphinxTranslator.dispatch_visit calling visit_desc_name for clib_versions SphinxTranslator.dispatch_visit calling visit_Text for clib_versions SphinxTranslator.dispatch_departure calling depart_Text for clib_versions SphinxTranslator.dispatch_departure calling depart_desc_name for clib_versions SphinxTranslator.dispatch_departure calling depart_desc_signature for clib_versions SphinxTranslator.dispatch_visit calling visit_desc_content for A map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_visit calling visit_paragraph for A map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_visit calling visit_Text for A map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_departure calling depart_Text for A map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_departure calling depart_paragraph for A map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_departure calling depart_desc_content for A map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_departure calling depart_desc for clib_versionsA map for the versions of the compression libraries included in C library. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for cnamesThe list of compressors included in C library. SphinxTranslator.dispatch_visit calling visit_desc_signature for cnames SphinxTranslator.dispatch_visit calling visit_desc_name for cnames SphinxTranslator.dispatch_visit calling visit_Text for cnames SphinxTranslator.dispatch_departure calling depart_Text for cnames SphinxTranslator.dispatch_departure calling depart_desc_name for cnames SphinxTranslator.dispatch_departure calling depart_desc_signature for cnames SphinxTranslator.dispatch_visit calling visit_desc_content for The list of compressors included in C library. SphinxTranslator.dispatch_visit calling visit_paragraph for The list of compressors included in C library. SphinxTranslator.dispatch_visit calling visit_Text for The list of compressors included in C library. SphinxTranslator.dispatch_departure calling depart_Text for The list of compressors included in C library. SphinxTranslator.dispatch_departure calling depart_paragraph for The list of compressors included in C library. SphinxTranslator.dispatch_departure calling depart_desc_content for The list of compressors included in C library. SphinxTranslator.dispatch_departure calling depart_desc for cnamesThe list of compressors included in C library. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for cname2clibA map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_visit calling visit_desc_signature for cname2clib SphinxTranslator.dispatch_visit calling visit_desc_name for cname2clib SphinxTranslator.dispatch_visit calling visit_Text for cname2clib SphinxTranslator.dispatch_departure calling depart_Text for cname2clib SphinxTranslator.dispatch_departure calling depart_desc_name for cname2clib SphinxTranslator.dispatch_departure calling depart_desc_signature for cname2clib SphinxTranslator.dispatch_visit calling visit_desc_content for A map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_visit calling visit_paragraph for A map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_visit calling visit_Text for A map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_departure calling depart_Text for A map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_departure calling depart_paragraph for A map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_departure calling depart_desc_content for A map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_departure calling depart_desc for cname2clibA map between compressor names and its libraries (or formats). SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for ncoresThe number of cores detected. SphinxTranslator.dispatch_visit calling visit_desc_signature for ncores SphinxTranslator.dispatch_visit calling visit_desc_name for ncores SphinxTranslator.dispatch_visit calling visit_Text for ncores SphinxTranslator.dispatch_departure calling depart_Text for ncores SphinxTranslator.dispatch_departure calling depart_desc_name for ncores SphinxTranslator.dispatch_departure calling depart_desc_signature for ncores SphinxTranslator.dispatch_visit calling visit_desc_content for The number of cores detected. SphinxTranslator.dispatch_visit calling visit_paragraph for The number of cores detected. SphinxTranslator.dispatch_visit calling visit_Text for The number of cores detected. SphinxTranslator.dispatch_departure calling depart_Text for The number of cores detected. SphinxTranslator.dispatch_departure calling depart_paragraph for The number of cores detected. SphinxTranslator.dispatch_departure calling depart_desc_content for The number of cores detected. SphinxTranslator.dispatch_departure calling depart_desc for ncoresThe number of cores detected. SphinxTranslator.dispatch_departure calling depart_section for
First level variables__version__The version of the blosc package.blosclib_versionThe version of the Blosc C library.clib_versionsA map for the versions of the compression libraries included in C library.cnamesThe list of compressors included in C library.cname2clibA map between compressor names and its libraries (or formats).ncoresThe number of cores detected.
SphinxTranslator.dispatch_visit calling visit_section for
Public functionsblosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) Trueblosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans Trueblosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray Trueblosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize Trueblosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize Trueblosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True
SphinxTranslator.dispatch_visit calling visit_title for Public functions SphinxTranslator.dispatch_visit calling visit_Text for Public functions SphinxTranslator.dispatch_departure calling depart_Text for Public functions SphinxTranslator.dispatch_departure calling depart_title for Public functions SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) True SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for compress SphinxTranslator.dispatch_visit calling visit_Text for compress SphinxTranslator.dispatch_departure calling depart_Text for compress SphinxTranslator.dispatch_departure calling depart_desc_name for compress SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for bytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_parameter for bytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_Text for bytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_Text for bytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_parameter for bytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for bytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_content for Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) True SphinxTranslator.dispatch_visit calling visit_paragraph for Compress bytesobj, with a given type size. SphinxTranslator.dispatch_visit calling visit_Text for Compress bytesobj, with a given type size. SphinxTranslator.dispatch_departure calling depart_Text for Compress bytesobj, with a given type size. SphinxTranslator.dispatch_departure calling depart_paragraph for Compress bytesobj, with a given type size. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_visit calling visit_field for Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for bytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_paragraph for bytesobj : bytes-like object (supporting the buffer interface) SphinxTranslator.dispatch_visit calling visit_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for : bytes-like object (supporting the buffer interface) SphinxTranslator.dispatch_departure calling depart_Text for : bytes-like object (supporting the buffer interface) SphinxTranslator.dispatch_departure calling depart_paragraph for bytesobj : bytes-like object (supporting the buffer interface) SphinxTranslator.dispatch_visit calling visit_block_quote for The data to be compressed. SphinxTranslator.dispatch_visit calling visit_paragraph for The data to be compressed. SphinxTranslator.dispatch_visit calling visit_Text for The data to be compressed. SphinxTranslator.dispatch_departure calling depart_Text for The data to be compressed. SphinxTranslator.dispatch_departure calling depart_paragraph for The data to be compressed. SphinxTranslator.dispatch_departure calling depart_block_quote for The data to be compressed. SphinxTranslator.dispatch_visit calling visit_paragraph for typesize : int SphinxTranslator.dispatch_visit calling visit_strong for typesize SphinxTranslator.dispatch_visit calling visit_Text for typesize SphinxTranslator.dispatch_departure calling depart_Text for typesize SphinxTranslator.dispatch_departure calling depart_strong for typesize SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for typesize : int SphinxTranslator.dispatch_visit calling visit_block_quote for The data type size. SphinxTranslator.dispatch_visit calling visit_paragraph for The data type size. SphinxTranslator.dispatch_visit calling visit_Text for The data type size. SphinxTranslator.dispatch_departure calling depart_Text for The data type size. SphinxTranslator.dispatch_departure calling depart_paragraph for The data type size. SphinxTranslator.dispatch_departure calling depart_block_quote for The data type size. SphinxTranslator.dispatch_visit calling visit_paragraph for clevel : int (optional) SphinxTranslator.dispatch_visit calling visit_strong for clevel SphinxTranslator.dispatch_visit calling visit_Text for clevel SphinxTranslator.dispatch_departure calling depart_Text for clevel SphinxTranslator.dispatch_departure calling depart_strong for clevel SphinxTranslator.dispatch_visit calling visit_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for clevel : int (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_paragraph for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_Text for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_Text for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_paragraph for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_block_quote for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_paragraph for shuffle : int (optional) SphinxTranslator.dispatch_visit calling visit_strong for shuffle SphinxTranslator.dispatch_visit calling visit_Text for shuffle SphinxTranslator.dispatch_departure calling depart_Text for shuffle SphinxTranslator.dispatch_departure calling depart_strong for shuffle SphinxTranslator.dispatch_visit calling visit_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for shuffle : int (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_paragraph for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_Text for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_Text for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_paragraph for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_block_quote for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_paragraph for cname : string (optional) SphinxTranslator.dispatch_visit calling visit_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for cname SphinxTranslator.dispatch_departure calling depart_Text for cname SphinxTranslator.dispatch_departure calling depart_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for : string (optional) SphinxTranslator.dispatch_departure calling depart_Text for : string (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for cname : string (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_paragraph for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_Text for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_Text for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_paragraph for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_block_quote for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_field_body for bytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_field for Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for out : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for out : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_Text for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_Text for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_paragraph for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_block_quote for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_field_body for out : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_field for RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for TypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_visit calling visit_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_strong for TypeError SphinxTranslator.dispatch_visit calling visit_Text for TypeError SphinxTranslator.dispatch_departure calling depart_Text for TypeError SphinxTranslator.dispatch_departure calling depart_strong for TypeError SphinxTranslator.dispatch_departure calling depart_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_block_quote for If bytesobj doesn’t support the buffer interface. SphinxTranslator.dispatch_visit calling visit_paragraph for If bytesobj doesn’t support the buffer interface. SphinxTranslator.dispatch_visit calling visit_Text for If bytesobj doesn’t support the buffer interface. SphinxTranslator.dispatch_departure calling depart_Text for If bytesobj doesn’t support the buffer interface. SphinxTranslator.dispatch_departure calling depart_paragraph for If bytesobj doesn’t support the buffer interface. SphinxTranslator.dispatch_departure calling depart_block_quote for If bytesobj doesn’t support the buffer interface. SphinxTranslator.dispatch_visit calling visit_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_strong for ValueError SphinxTranslator.dispatch_visit calling visit_Text for ValueError SphinxTranslator.dispatch_departure calling depart_Text for ValueError SphinxTranslator.dispatch_departure calling depart_strong for ValueError SphinxTranslator.dispatch_departure calling depart_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_block_quote for If bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_visit calling visit_paragraph for If bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_visit calling visit_Text for If bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_departure calling depart_Text for If bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_departure calling depart_paragraph for If bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_departure calling depart_block_quote for If bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_departure calling depart_field_body for TypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_departure calling depart_field for RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) True SphinxTranslator.dispatch_departure calling depart_desc_content for Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) True SphinxTranslator.dispatch_departure calling depart_desc for blosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) True SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for compress_ptr SphinxTranslator.dispatch_visit calling visit_Text for compress_ptr SphinxTranslator.dispatch_departure calling depart_Text for compress_ptr SphinxTranslator.dispatch_departure calling depart_desc_name for compress_ptr SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for address, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_parameter for address, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_Text for address, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_Text for address, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_parameter for address, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for address, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_content for Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True SphinxTranslator.dispatch_visit calling visit_paragraph for Compress the data at address with given items and typesize. SphinxTranslator.dispatch_visit calling visit_Text for Compress the data at address with given items and typesize. SphinxTranslator.dispatch_departure calling depart_Text for Compress the data at address with given items and typesize. SphinxTranslator.dispatch_departure calling depart_paragraph for Compress the data at address with given items and typesize. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_field for Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for address : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_paragraph for address : int or long SphinxTranslator.dispatch_visit calling visit_strong for address SphinxTranslator.dispatch_visit calling visit_Text for address SphinxTranslator.dispatch_departure calling depart_Text for address SphinxTranslator.dispatch_departure calling depart_strong for address SphinxTranslator.dispatch_visit calling visit_Text for : int or long SphinxTranslator.dispatch_departure calling depart_Text for : int or long SphinxTranslator.dispatch_departure calling depart_paragraph for address : int or long SphinxTranslator.dispatch_visit calling visit_block_quote for the pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_paragraph for the pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_Text for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_Text for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_paragraph for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_block_quote for the pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_paragraph for items : int SphinxTranslator.dispatch_visit calling visit_strong for items SphinxTranslator.dispatch_visit calling visit_Text for items SphinxTranslator.dispatch_departure calling depart_Text for items SphinxTranslator.dispatch_departure calling depart_strong for items SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for items : int SphinxTranslator.dispatch_visit calling visit_block_quote for The number of items (of typesize) to be compressed. SphinxTranslator.dispatch_visit calling visit_paragraph for The number of items (of typesize) to be compressed. SphinxTranslator.dispatch_visit calling visit_Text for The number of items (of typesize) to be compressed. SphinxTranslator.dispatch_departure calling depart_Text for The number of items (of typesize) to be compressed. SphinxTranslator.dispatch_departure calling depart_paragraph for The number of items (of typesize) to be compressed. SphinxTranslator.dispatch_departure calling depart_block_quote for The number of items (of typesize) to be compressed. SphinxTranslator.dispatch_visit calling visit_paragraph for typesize : int SphinxTranslator.dispatch_visit calling visit_strong for typesize SphinxTranslator.dispatch_visit calling visit_Text for typesize SphinxTranslator.dispatch_departure calling depart_Text for typesize SphinxTranslator.dispatch_departure calling depart_strong for typesize SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for typesize : int SphinxTranslator.dispatch_visit calling visit_block_quote for The data type size. SphinxTranslator.dispatch_visit calling visit_paragraph for The data type size. SphinxTranslator.dispatch_visit calling visit_Text for The data type size. SphinxTranslator.dispatch_departure calling depart_Text for The data type size. SphinxTranslator.dispatch_departure calling depart_paragraph for The data type size. SphinxTranslator.dispatch_departure calling depart_block_quote for The data type size. SphinxTranslator.dispatch_visit calling visit_paragraph for clevel : int (optional) SphinxTranslator.dispatch_visit calling visit_strong for clevel SphinxTranslator.dispatch_visit calling visit_Text for clevel SphinxTranslator.dispatch_departure calling depart_Text for clevel SphinxTranslator.dispatch_departure calling depart_strong for clevel SphinxTranslator.dispatch_visit calling visit_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for clevel : int (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_paragraph for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_Text for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_Text for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_paragraph for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_block_quote for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_paragraph for shuffle : int (optional) SphinxTranslator.dispatch_visit calling visit_strong for shuffle SphinxTranslator.dispatch_visit calling visit_Text for shuffle SphinxTranslator.dispatch_departure calling depart_Text for shuffle SphinxTranslator.dispatch_departure calling depart_strong for shuffle SphinxTranslator.dispatch_visit calling visit_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for shuffle : int (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_paragraph for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_Text for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_Text for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_paragraph for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_block_quote for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_paragraph for cname : string (optional) SphinxTranslator.dispatch_visit calling visit_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for cname SphinxTranslator.dispatch_departure calling depart_Text for cname SphinxTranslator.dispatch_departure calling depart_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for : string (optional) SphinxTranslator.dispatch_departure calling depart_Text for : string (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for cname : string (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_paragraph for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_Text for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_Text for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_paragraph for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_block_quote for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_field_body for address : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_field for Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for out : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for out : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_Text for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_Text for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_paragraph for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_block_quote for The compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_field_body for out : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : str / bytesThe compressed data in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_field for RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for TypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_strong for TypeError SphinxTranslator.dispatch_visit calling visit_Text for TypeError SphinxTranslator.dispatch_departure calling depart_Text for TypeError SphinxTranslator.dispatch_departure calling depart_strong for TypeError SphinxTranslator.dispatch_departure calling depart_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_block_quote for If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_paragraph for If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_Text for If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_Text for If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_paragraph for If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_block_quote for If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_strong for ValueError SphinxTranslator.dispatch_visit calling visit_Text for ValueError SphinxTranslator.dispatch_departure calling depart_Text for ValueError SphinxTranslator.dispatch_departure calling depart_strong for ValueError SphinxTranslator.dispatch_departure calling depart_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_block_quote for If items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_paragraph for If items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_Text for If items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_Text for If items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_paragraph for If items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_block_quote for If items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_field_body for TypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_field for RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_rubric for Notes SphinxTranslator.dispatch_visit calling visit_Text for Notes SphinxTranslator.dispatch_departure calling depart_Text for Notes SphinxTranslator.dispatch_departure calling depart_rubric for Notes SphinxTranslator.dispatch_visit calling visit_paragraph for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_visit calling visit_Text for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_departure calling depart_Text for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_departure calling depart_paragraph for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_visit calling visit_paragraph for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_visit calling visit_Text for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_departure calling depart_Text for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_departure calling depart_paragraph for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True SphinxTranslator.dispatch_departure calling depart_desc_content for Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True SphinxTranslator.dispatch_departure calling depart_desc for blosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray True SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.decompressbytesobj SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for decompress SphinxTranslator.dispatch_visit calling visit_Text for decompress SphinxTranslator.dispatch_departure calling depart_Text for decompress SphinxTranslator.dispatch_departure calling depart_desc_name for decompress SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for bytesobj SphinxTranslator.dispatch_visit calling visit_desc_parameter for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_parameter for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.decompressbytesobj SphinxTranslator.dispatch_visit calling visit_desc_content for Decompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray True SphinxTranslator.dispatch_visit calling visit_paragraph for Decompresses a bytesobj compressed object. SphinxTranslator.dispatch_visit calling visit_Text for Decompresses a bytesobj compressed object. SphinxTranslator.dispatch_departure calling depart_Text for Decompresses a bytesobj compressed object. SphinxTranslator.dispatch_departure calling depart_paragraph for Decompresses a bytesobj compressed object. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_field for Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for bytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_visit calling visit_paragraph for bytesobj : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for bytesobj : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The data to be decompressed. SphinxTranslator.dispatch_visit calling visit_paragraph for The data to be decompressed. SphinxTranslator.dispatch_visit calling visit_Text for The data to be decompressed. SphinxTranslator.dispatch_departure calling depart_Text for The data to be decompressed. SphinxTranslator.dispatch_departure calling depart_paragraph for The data to be decompressed. SphinxTranslator.dispatch_departure calling depart_block_quote for The data to be decompressed. SphinxTranslator.dispatch_visit calling visit_paragraph for as_bytearray : bool, optional SphinxTranslator.dispatch_visit calling visit_strong for as_bytearray SphinxTranslator.dispatch_visit calling visit_Text for as_bytearray SphinxTranslator.dispatch_departure calling depart_Text for as_bytearray SphinxTranslator.dispatch_departure calling depart_strong for as_bytearray SphinxTranslator.dispatch_visit calling visit_Text for : bool, optional SphinxTranslator.dispatch_departure calling depart_Text for : bool, optional SphinxTranslator.dispatch_departure calling depart_paragraph for as_bytearray : bool, optional SphinxTranslator.dispatch_visit calling visit_block_quote for If this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_visit calling visit_paragraph for If this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_visit calling visit_Text for If this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_departure calling depart_Text for If this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_departure calling depart_paragraph for If this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_departure calling depart_block_quote for If this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_departure calling depart_field_body for bytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_departure calling depart_field for Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for out : str / bytes or bytearray SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes or bytearray SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes or bytearray SphinxTranslator.dispatch_departure calling depart_paragraph for out : str / bytes or bytearray SphinxTranslator.dispatch_visit calling visit_block_quote for The decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for The decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_visit calling visit_Text for The decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_departure calling depart_Text for The decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_departure calling depart_paragraph for The decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_departure calling depart_block_quote for The decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_departure calling depart_field_body for out : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object. SphinxTranslator.dispatch_visit calling visit_field for RaisesTypeErrorIf bytesobj is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for TypeErrorIf bytesobj is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_strong for TypeError SphinxTranslator.dispatch_visit calling visit_Text for TypeError SphinxTranslator.dispatch_departure calling depart_Text for TypeError SphinxTranslator.dispatch_departure calling depart_strong for TypeError SphinxTranslator.dispatch_departure calling depart_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_block_quote for If bytesobj is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_paragraph for If bytesobj is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_Text for If bytesobj is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_Text for If bytesobj is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_paragraph for If bytesobj is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_block_quote for If bytesobj is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_field_body for TypeErrorIf bytesobj is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_field for RaisesTypeErrorIf bytesobj is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray True SphinxTranslator.dispatch_departure calling depart_desc_content for Decompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray True SphinxTranslator.dispatch_departure calling depart_desc for blosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray True SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize True SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.decompress_ptrbytesobjaddress SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for decompress_ptr SphinxTranslator.dispatch_visit calling visit_Text for decompress_ptr SphinxTranslator.dispatch_departure calling depart_Text for decompress_ptr SphinxTranslator.dispatch_departure calling depart_desc_name for decompress_ptr SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for bytesobjaddress SphinxTranslator.dispatch_visit calling visit_desc_parameter for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_parameter for bytesobj SphinxTranslator.dispatch_visit calling visit_desc_parameter for address SphinxTranslator.dispatch_visit calling visit_Text for address SphinxTranslator.dispatch_departure calling depart_Text for address SphinxTranslator.dispatch_departure calling depart_desc_parameter for address SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for bytesobjaddress SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.decompress_ptrbytesobjaddress SphinxTranslator.dispatch_visit calling visit_desc_content for Decompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize True SphinxTranslator.dispatch_visit calling visit_paragraph for Decompresses a bytesobj compressed object into the memory at address. SphinxTranslator.dispatch_visit calling visit_Text for Decompresses a bytesobj compressed object into the memory at address. SphinxTranslator.dispatch_departure calling depart_Text for Decompresses a bytesobj compressed object into the memory at address. SphinxTranslator.dispatch_departure calling depart_paragraph for Decompresses a bytesobj compressed object into the memory at address. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_field for Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for bytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_paragraph for bytesobj : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for bytesobj : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The data to be decompressed. SphinxTranslator.dispatch_visit calling visit_paragraph for The data to be decompressed. SphinxTranslator.dispatch_visit calling visit_Text for The data to be decompressed. SphinxTranslator.dispatch_departure calling depart_Text for The data to be decompressed. SphinxTranslator.dispatch_departure calling depart_paragraph for The data to be decompressed. SphinxTranslator.dispatch_departure calling depart_block_quote for The data to be decompressed. SphinxTranslator.dispatch_visit calling visit_paragraph for address : int or long SphinxTranslator.dispatch_visit calling visit_strong for address SphinxTranslator.dispatch_visit calling visit_Text for address SphinxTranslator.dispatch_departure calling depart_Text for address SphinxTranslator.dispatch_departure calling depart_strong for address SphinxTranslator.dispatch_visit calling visit_Text for : int or long SphinxTranslator.dispatch_departure calling depart_Text for : int or long SphinxTranslator.dispatch_departure calling depart_paragraph for address : int or long SphinxTranslator.dispatch_visit calling visit_block_quote for the pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_paragraph for the pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_Text for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_Text for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_paragraph for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_block_quote for the pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_field_body for bytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressed SphinxTranslator.dispatch_departure calling depart_field for Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressed SphinxTranslator.dispatch_visit calling visit_field for Returnsnbytes : intthe number of bytes written to the buffer SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for nbytes : intthe number of bytes written to the buffer SphinxTranslator.dispatch_visit calling visit_paragraph for nbytes : int SphinxTranslator.dispatch_visit calling visit_strong for nbytes SphinxTranslator.dispatch_visit calling visit_Text for nbytes SphinxTranslator.dispatch_departure calling depart_Text for nbytes SphinxTranslator.dispatch_departure calling depart_strong for nbytes SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for nbytes : int SphinxTranslator.dispatch_visit calling visit_block_quote for the number of bytes written to the buffer SphinxTranslator.dispatch_visit calling visit_paragraph for the number of bytes written to the buffer SphinxTranslator.dispatch_visit calling visit_Text for the number of bytes written to the buffer SphinxTranslator.dispatch_departure calling depart_Text for the number of bytes written to the buffer SphinxTranslator.dispatch_departure calling depart_paragraph for the number of bytes written to the buffer SphinxTranslator.dispatch_departure calling depart_block_quote for the number of bytes written to the buffer SphinxTranslator.dispatch_departure calling depart_field_body for nbytes : intthe number of bytes written to the buffer SphinxTranslator.dispatch_departure calling depart_field for Returnsnbytes : intthe number of bytes written to the buffer SphinxTranslator.dispatch_visit calling visit_field for RaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for TypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_strong for TypeError SphinxTranslator.dispatch_visit calling visit_Text for TypeError SphinxTranslator.dispatch_departure calling depart_Text for TypeError SphinxTranslator.dispatch_departure calling depart_strong for TypeError SphinxTranslator.dispatch_departure calling depart_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_block_quote for If bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_paragraph for If bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_Text for If bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_Text for If bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_paragraph for If bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_block_quote for If bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_field_body for TypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_field for RaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long. SphinxTranslator.dispatch_visit calling visit_rubric for Notes SphinxTranslator.dispatch_visit calling visit_Text for Notes SphinxTranslator.dispatch_departure calling depart_Text for Notes SphinxTranslator.dispatch_departure calling depart_rubric for Notes SphinxTranslator.dispatch_visit calling visit_paragraph for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_visit calling visit_Text for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_departure calling depart_Text for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_departure calling depart_paragraph for This function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules. SphinxTranslator.dispatch_visit calling visit_paragraph for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_visit calling visit_Text for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_departure calling depart_Text for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_departure calling depart_paragraph for Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize True SphinxTranslator.dispatch_departure calling depart_desc_content for Decompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize True SphinxTranslator.dispatch_departure calling depart_desc for blosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize True SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for pack_array SphinxTranslator.dispatch_visit calling visit_Text for pack_array SphinxTranslator.dispatch_departure calling depart_Text for pack_array SphinxTranslator.dispatch_departure calling depart_desc_name for pack_array SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for array[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_parameter for array[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_Text for array[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_Text for array[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_parameter for array[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for array[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']] SphinxTranslator.dispatch_visit calling visit_desc_content for Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True SphinxTranslator.dispatch_visit calling visit_paragraph for Pack (compress) a NumPy array. SphinxTranslator.dispatch_visit calling visit_Text for Pack (compress) a NumPy array. SphinxTranslator.dispatch_departure calling depart_Text for Pack (compress) a NumPy array. SphinxTranslator.dispatch_departure calling depart_paragraph for Pack (compress) a NumPy array. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_field for Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for array : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_paragraph for array : ndarray SphinxTranslator.dispatch_visit calling visit_strong for array SphinxTranslator.dispatch_visit calling visit_Text for array SphinxTranslator.dispatch_departure calling depart_Text for array SphinxTranslator.dispatch_departure calling depart_strong for array SphinxTranslator.dispatch_visit calling visit_Text for : ndarray SphinxTranslator.dispatch_departure calling depart_Text for : ndarray SphinxTranslator.dispatch_departure calling depart_paragraph for array : ndarray SphinxTranslator.dispatch_visit calling visit_block_quote for The NumPy array to be packed. SphinxTranslator.dispatch_visit calling visit_paragraph for The NumPy array to be packed. SphinxTranslator.dispatch_visit calling visit_Text for The NumPy array to be packed. SphinxTranslator.dispatch_departure calling depart_Text for The NumPy array to be packed. SphinxTranslator.dispatch_departure calling depart_paragraph for The NumPy array to be packed. SphinxTranslator.dispatch_departure calling depart_block_quote for The NumPy array to be packed. SphinxTranslator.dispatch_visit calling visit_paragraph for clevel : int (optional) SphinxTranslator.dispatch_visit calling visit_strong for clevel SphinxTranslator.dispatch_visit calling visit_Text for clevel SphinxTranslator.dispatch_departure calling depart_Text for clevel SphinxTranslator.dispatch_departure calling depart_strong for clevel SphinxTranslator.dispatch_visit calling visit_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for clevel : int (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_paragraph for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_Text for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_Text for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_paragraph for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_departure calling depart_block_quote for The compression level from 0 (no compression) to 9 (maximum compression). The default is 9. SphinxTranslator.dispatch_visit calling visit_paragraph for shuffle : int (optional) SphinxTranslator.dispatch_visit calling visit_strong for shuffle SphinxTranslator.dispatch_visit calling visit_Text for shuffle SphinxTranslator.dispatch_departure calling depart_Text for shuffle SphinxTranslator.dispatch_departure calling depart_strong for shuffle SphinxTranslator.dispatch_visit calling visit_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_Text for : int (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for shuffle : int (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_paragraph for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_Text for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_Text for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_paragraph for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_departure calling depart_block_quote for The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE. SphinxTranslator.dispatch_visit calling visit_paragraph for cname : string (optional) SphinxTranslator.dispatch_visit calling visit_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for cname SphinxTranslator.dispatch_departure calling depart_Text for cname SphinxTranslator.dispatch_departure calling depart_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for : string (optional) SphinxTranslator.dispatch_departure calling depart_Text for : string (optional) SphinxTranslator.dispatch_departure calling depart_paragraph for cname : string (optional) SphinxTranslator.dispatch_visit calling visit_block_quote for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_paragraph for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_Text for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_Text for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_paragraph for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_block_quote for The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_field_body for array : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_departure calling depart_field for Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : str / bytesThe packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : str / bytesThe packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for out : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for out : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_paragraph for The packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_Text for The packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_Text for The packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_paragraph for The packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_block_quote for The packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_field_body for out : str / bytesThe packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : str / bytesThe packed array in form of a Python str / bytes object. SphinxTranslator.dispatch_visit calling visit_field for RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for TypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_strong for TypeError SphinxTranslator.dispatch_visit calling visit_Text for TypeError SphinxTranslator.dispatch_departure calling depart_Text for TypeError SphinxTranslator.dispatch_departure calling depart_strong for TypeError SphinxTranslator.dispatch_departure calling depart_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_block_quote for If array does not quack like a numpy ndarray. SphinxTranslator.dispatch_visit calling visit_paragraph for If array does not quack like a numpy ndarray. SphinxTranslator.dispatch_visit calling visit_Text for If array does not quack like a numpy ndarray. SphinxTranslator.dispatch_departure calling depart_Text for If array does not quack like a numpy ndarray. SphinxTranslator.dispatch_departure calling depart_paragraph for If array does not quack like a numpy ndarray. SphinxTranslator.dispatch_departure calling depart_block_quote for If array does not quack like a numpy ndarray. SphinxTranslator.dispatch_visit calling visit_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_strong for ValueError SphinxTranslator.dispatch_visit calling visit_Text for ValueError SphinxTranslator.dispatch_departure calling depart_Text for ValueError SphinxTranslator.dispatch_departure calling depart_strong for ValueError SphinxTranslator.dispatch_departure calling depart_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_block_quote for If array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_definition_list for If array.itemsize * array.size is larger than the maximum allowedbuffer size. SphinxTranslator.dispatch_visit calling visit_definition_list_item for If array.itemsize * array.size is larger than the maximum allowedbuffer size. SphinxTranslator.dispatch_visit calling visit_term for If array.itemsize * array.size is larger than the maximum allowed SphinxTranslator.dispatch_visit calling visit_Text for If array.itemsize * array.size is larger than the maximum allowed SphinxTranslator.dispatch_departure calling depart_Text for If array.itemsize * array.size is larger than the maximum allowed SphinxTranslator.dispatch_departure calling depart_term for If array.itemsize * array.size is larger than the maximum allowed SphinxTranslator.dispatch_visit calling visit_definition for buffer size. SphinxTranslator.dispatch_visit calling visit_paragraph for buffer size. SphinxTranslator.dispatch_visit calling visit_Text for buffer size. SphinxTranslator.dispatch_departure calling depart_Text for buffer size. SphinxTranslator.dispatch_departure calling depart_paragraph for buffer size. SphinxTranslator.dispatch_departure calling depart_definition for buffer size. SphinxTranslator.dispatch_departure calling depart_definition_list_item for If array.itemsize * array.size is larger than the maximum allowedbuffer size. SphinxTranslator.dispatch_departure calling depart_definition_list for If array.itemsize * array.size is larger than the maximum allowedbuffer size. SphinxTranslator.dispatch_visit calling visit_paragraph for If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_Text for If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_Text for If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_paragraph for If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_block_quote for If array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_field_body for TypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_field for RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True SphinxTranslator.dispatch_departure calling depart_desc_content for Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True SphinxTranslator.dispatch_departure calling depart_desc for blosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.unpack_arraypacked_array SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for unpack_array SphinxTranslator.dispatch_visit calling visit_Text for unpack_array SphinxTranslator.dispatch_departure calling depart_Text for unpack_array SphinxTranslator.dispatch_departure calling depart_desc_name for unpack_array SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for packed_array SphinxTranslator.dispatch_visit calling visit_desc_parameter for packed_array SphinxTranslator.dispatch_visit calling visit_Text for packed_array SphinxTranslator.dispatch_departure calling depart_Text for packed_array SphinxTranslator.dispatch_departure calling depart_desc_parameter for packed_array SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for packed_array SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.unpack_arraypacked_array SphinxTranslator.dispatch_visit calling visit_desc_content for Unpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True SphinxTranslator.dispatch_visit calling visit_paragraph for Unpack (decompress) a packed NumPy array. SphinxTranslator.dispatch_visit calling visit_Text for Unpack (decompress) a packed NumPy array. SphinxTranslator.dispatch_departure calling depart_Text for Unpack (decompress) a packed NumPy array. SphinxTranslator.dispatch_departure calling depart_paragraph for Unpack (decompress) a packed NumPy array. SphinxTranslator.dispatch_visit calling visit_field_list for Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_field for Parameterspacked_array : str / bytesThe packed array to be decompressed. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for packed_array : str / bytesThe packed array to be decompressed. SphinxTranslator.dispatch_visit calling visit_paragraph for packed_array : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for packed_array SphinxTranslator.dispatch_visit calling visit_Text for packed_array SphinxTranslator.dispatch_departure calling depart_Text for packed_array SphinxTranslator.dispatch_departure calling depart_strong for packed_array SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for packed_array : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The packed array to be decompressed. SphinxTranslator.dispatch_visit calling visit_paragraph for The packed array to be decompressed. SphinxTranslator.dispatch_visit calling visit_Text for The packed array to be decompressed. SphinxTranslator.dispatch_departure calling depart_Text for The packed array to be decompressed. SphinxTranslator.dispatch_departure calling depart_paragraph for The packed array to be decompressed. SphinxTranslator.dispatch_departure calling depart_block_quote for The packed array to be decompressed. SphinxTranslator.dispatch_departure calling depart_field_body for packed_array : str / bytesThe packed array to be decompressed. SphinxTranslator.dispatch_departure calling depart_field for Parameterspacked_array : str / bytesThe packed array to be decompressed. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : ndarrayThe decompressed data in form of a NumPy array. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : ndarrayThe decompressed data in form of a NumPy array. SphinxTranslator.dispatch_visit calling visit_paragraph for out : ndarray SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : ndarray SphinxTranslator.dispatch_departure calling depart_Text for : ndarray SphinxTranslator.dispatch_departure calling depart_paragraph for out : ndarray SphinxTranslator.dispatch_visit calling visit_block_quote for The decompressed data in form of a NumPy array. SphinxTranslator.dispatch_visit calling visit_paragraph for The decompressed data in form of a NumPy array. SphinxTranslator.dispatch_visit calling visit_Text for The decompressed data in form of a NumPy array. SphinxTranslator.dispatch_departure calling depart_Text for The decompressed data in form of a NumPy array. SphinxTranslator.dispatch_departure calling depart_paragraph for The decompressed data in form of a NumPy array. SphinxTranslator.dispatch_departure calling depart_block_quote for The decompressed data in form of a NumPy array. SphinxTranslator.dispatch_departure calling depart_field_body for out : ndarrayThe decompressed data in form of a NumPy array. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : ndarrayThe decompressed data in form of a NumPy array. SphinxTranslator.dispatch_visit calling visit_field for RaisesTypeErrorIf packed_array is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for TypeErrorIf packed_array is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_strong for TypeError SphinxTranslator.dispatch_visit calling visit_Text for TypeError SphinxTranslator.dispatch_departure calling depart_Text for TypeError SphinxTranslator.dispatch_departure calling depart_strong for TypeError SphinxTranslator.dispatch_departure calling depart_paragraph for TypeError SphinxTranslator.dispatch_visit calling visit_block_quote for If packed_array is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_paragraph for If packed_array is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_Text for If packed_array is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_Text for If packed_array is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_paragraph for If packed_array is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_block_quote for If packed_array is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_field_body for TypeErrorIf packed_array is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_field for RaisesTypeErrorIf packed_array is not of type bytes or string. SphinxTranslator.dispatch_departure calling depart_field_list for Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True SphinxTranslator.dispatch_departure calling depart_desc_content for Unpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True SphinxTranslator.dispatch_departure calling depart_desc for blosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True SphinxTranslator.dispatch_departure calling depart_section for
Public functionsblosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) Trueblosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans Trueblosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray Trueblosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize Trueblosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize Trueblosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True
SphinxTranslator.dispatch_visit calling visit_section for
Utilitiesblosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version.blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names.blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system.blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>>blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library.blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512)blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True)blosc.print_versionsPrint all the versions of software that python-blosc relies on.
SphinxTranslator.dispatch_visit calling visit_title for Utilities SphinxTranslator.dispatch_visit calling visit_Text for Utilities SphinxTranslator.dispatch_departure calling depart_Text for Utilities SphinxTranslator.dispatch_departure calling depart_title for Utilities SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.clib_infocname SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for clib_info SphinxTranslator.dispatch_visit calling visit_Text for clib_info SphinxTranslator.dispatch_departure calling depart_Text for clib_info SphinxTranslator.dispatch_departure calling depart_desc_name for clib_info SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for cname SphinxTranslator.dispatch_visit calling visit_desc_parameter for cname SphinxTranslator.dispatch_visit calling visit_Text for cname SphinxTranslator.dispatch_departure calling depart_Text for cname SphinxTranslator.dispatch_departure calling depart_desc_parameter for cname SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for cname SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.clib_infocname SphinxTranslator.dispatch_visit calling visit_desc_content for Return info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_visit calling visit_paragraph for Return info for compression libraries in C library. SphinxTranslator.dispatch_visit calling visit_Text for Return info for compression libraries in C library. SphinxTranslator.dispatch_departure calling depart_Text for Return info for compression libraries in C library. SphinxTranslator.dispatch_departure calling depart_paragraph for Return info for compression libraries in C library. SphinxTranslator.dispatch_visit calling visit_field_list for Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_visit calling visit_field for Parameterscname : strThe compressor name. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for cname : strThe compressor name. SphinxTranslator.dispatch_visit calling visit_paragraph for cname : str SphinxTranslator.dispatch_visit calling visit_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for cname SphinxTranslator.dispatch_departure calling depart_Text for cname SphinxTranslator.dispatch_departure calling depart_strong for cname SphinxTranslator.dispatch_visit calling visit_Text for : str SphinxTranslator.dispatch_departure calling depart_Text for : str SphinxTranslator.dispatch_departure calling depart_paragraph for cname : str SphinxTranslator.dispatch_visit calling visit_block_quote for The compressor name. SphinxTranslator.dispatch_visit calling visit_paragraph for The compressor name. SphinxTranslator.dispatch_visit calling visit_Text for The compressor name. SphinxTranslator.dispatch_departure calling depart_Text for The compressor name. SphinxTranslator.dispatch_departure calling depart_paragraph for The compressor name. SphinxTranslator.dispatch_departure calling depart_block_quote for The compressor name. SphinxTranslator.dispatch_departure calling depart_field_body for cname : strThe compressor name. SphinxTranslator.dispatch_departure calling depart_field for Parameterscname : strThe compressor name. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : tupleThe associated library name and version. SphinxTranslator.dispatch_visit calling visit_paragraph for out : tuple SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : tuple SphinxTranslator.dispatch_departure calling depart_Text for : tuple SphinxTranslator.dispatch_departure calling depart_paragraph for out : tuple SphinxTranslator.dispatch_visit calling visit_block_quote for The associated library name and version. SphinxTranslator.dispatch_visit calling visit_paragraph for The associated library name and version. SphinxTranslator.dispatch_visit calling visit_Text for The associated library name and version. SphinxTranslator.dispatch_departure calling depart_Text for The associated library name and version. SphinxTranslator.dispatch_departure calling depart_paragraph for The associated library name and version. SphinxTranslator.dispatch_departure calling depart_block_quote for The associated library name and version. SphinxTranslator.dispatch_departure calling depart_field_body for out : tupleThe associated library name and version. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_departure calling depart_field_list for Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_departure calling depart_desc_content for Return info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_departure calling depart_desc for blosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names. SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.compressor_list SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for compressor_list SphinxTranslator.dispatch_visit calling visit_Text for compressor_list SphinxTranslator.dispatch_departure calling depart_Text for compressor_list SphinxTranslator.dispatch_departure calling depart_desc_name for compressor_list SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.compressor_list SphinxTranslator.dispatch_visit calling visit_desc_content for Returns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names. SphinxTranslator.dispatch_visit calling visit_paragraph for Returns a list of compressors available in C library. SphinxTranslator.dispatch_visit calling visit_Text for Returns a list of compressors available in C library. SphinxTranslator.dispatch_departure calling depart_Text for Returns a list of compressors available in C library. SphinxTranslator.dispatch_departure calling depart_paragraph for Returns a list of compressors available in C library. SphinxTranslator.dispatch_visit calling visit_field_list for ParametersNoneReturnsout : listThe list of names. SphinxTranslator.dispatch_visit calling visit_field for ParametersNone SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for None SphinxTranslator.dispatch_visit calling visit_paragraph for None SphinxTranslator.dispatch_visit calling visit_strong for None SphinxTranslator.dispatch_visit calling visit_Text for None SphinxTranslator.dispatch_departure calling depart_Text for None SphinxTranslator.dispatch_departure calling depart_strong for None SphinxTranslator.dispatch_departure calling depart_paragraph for None SphinxTranslator.dispatch_departure calling depart_field_body for None SphinxTranslator.dispatch_departure calling depart_field for ParametersNone SphinxTranslator.dispatch_visit calling visit_field for Returnsout : listThe list of names. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : listThe list of names. SphinxTranslator.dispatch_visit calling visit_paragraph for out : list SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : list SphinxTranslator.dispatch_departure calling depart_Text for : list SphinxTranslator.dispatch_departure calling depart_paragraph for out : list SphinxTranslator.dispatch_visit calling visit_block_quote for The list of names. SphinxTranslator.dispatch_visit calling visit_paragraph for The list of names. SphinxTranslator.dispatch_visit calling visit_Text for The list of names. SphinxTranslator.dispatch_departure calling depart_Text for The list of names. SphinxTranslator.dispatch_departure calling depart_paragraph for The list of names. SphinxTranslator.dispatch_departure calling depart_block_quote for The list of names. SphinxTranslator.dispatch_departure calling depart_field_body for out : listThe list of names. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : listThe list of names. SphinxTranslator.dispatch_departure calling depart_field_list for ParametersNoneReturnsout : listThe list of names. SphinxTranslator.dispatch_departure calling depart_desc_content for Returns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names. SphinxTranslator.dispatch_departure calling depart_desc for blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.detect_number_of_cores SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for detect_number_of_cores SphinxTranslator.dispatch_visit calling visit_Text for detect_number_of_cores SphinxTranslator.dispatch_departure calling depart_Text for detect_number_of_cores SphinxTranslator.dispatch_departure calling depart_desc_name for detect_number_of_cores SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.detect_number_of_cores SphinxTranslator.dispatch_visit calling visit_desc_content for Detect the number of cores in this system.Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_visit calling visit_paragraph for Detect the number of cores in this system. SphinxTranslator.dispatch_visit calling visit_Text for Detect the number of cores in this system. SphinxTranslator.dispatch_departure calling depart_Text for Detect the number of cores in this system. SphinxTranslator.dispatch_departure calling depart_paragraph for Detect the number of cores in this system. SphinxTranslator.dispatch_visit calling visit_field_list for Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : intThe number of cores in this system. SphinxTranslator.dispatch_visit calling visit_paragraph for out : int SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for out : int SphinxTranslator.dispatch_visit calling visit_block_quote for The number of cores in this system. SphinxTranslator.dispatch_visit calling visit_paragraph for The number of cores in this system. SphinxTranslator.dispatch_visit calling visit_Text for The number of cores in this system. SphinxTranslator.dispatch_departure calling depart_Text for The number of cores in this system. SphinxTranslator.dispatch_departure calling depart_paragraph for The number of cores in this system. SphinxTranslator.dispatch_departure calling depart_block_quote for The number of cores in this system. SphinxTranslator.dispatch_departure calling depart_field_body for out : intThe number of cores in this system. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_departure calling depart_field_list for Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_departure calling depart_desc_content for Detect the number of cores in this system.Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_departure calling depart_desc for blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>> SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.free_resources SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for free_resources SphinxTranslator.dispatch_visit calling visit_Text for free_resources SphinxTranslator.dispatch_departure calling depart_Text for free_resources SphinxTranslator.dispatch_departure calling depart_desc_name for free_resources SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.free_resources SphinxTranslator.dispatch_visit calling visit_desc_content for Free possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>> SphinxTranslator.dispatch_visit calling visit_paragraph for Free possible memory temporaries and thread resources. SphinxTranslator.dispatch_visit calling visit_Text for Free possible memory temporaries and thread resources. SphinxTranslator.dispatch_departure calling depart_Text for Free possible memory temporaries and thread resources. SphinxTranslator.dispatch_departure calling depart_paragraph for Free possible memory temporaries and thread resources. SphinxTranslator.dispatch_visit calling visit_field_list for Returnsout : None SphinxTranslator.dispatch_visit calling visit_field for Returnsout : None SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : None SphinxTranslator.dispatch_visit calling visit_paragraph for out : None SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : None SphinxTranslator.dispatch_departure calling depart_Text for : None SphinxTranslator.dispatch_departure calling depart_paragraph for out : None SphinxTranslator.dispatch_departure calling depart_field_body for out : None SphinxTranslator.dispatch_departure calling depart_field for Returnsout : None SphinxTranslator.dispatch_departure calling depart_field_list for Returnsout : None SphinxTranslator.dispatch_visit calling visit_rubric for Notes SphinxTranslator.dispatch_visit calling visit_Text for Notes SphinxTranslator.dispatch_departure calling depart_Text for Notes SphinxTranslator.dispatch_departure calling depart_rubric for Notes SphinxTranslator.dispatch_visit calling visit_paragraph for Blosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while. SphinxTranslator.dispatch_visit calling visit_Text for Blosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while. SphinxTranslator.dispatch_departure calling depart_Text for Blosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while. SphinxTranslator.dispatch_departure calling depart_paragraph for Blosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> blosc.free_resources() >>> SphinxTranslator.dispatch_departure calling depart_desc_content for Free possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>> SphinxTranslator.dispatch_departure calling depart_desc for blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>> SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.get_clibbytesobj SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for get_clib SphinxTranslator.dispatch_visit calling visit_Text for get_clib SphinxTranslator.dispatch_departure calling depart_Text for get_clib SphinxTranslator.dispatch_departure calling depart_desc_name for get_clib SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for bytesobj SphinxTranslator.dispatch_visit calling visit_desc_parameter for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_parameter for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for bytesobj SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.get_clibbytesobj SphinxTranslator.dispatch_visit calling visit_desc_content for Return the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_visit calling visit_paragraph for Return the name of the compression library for Blosc bytesobj buffer. SphinxTranslator.dispatch_visit calling visit_Text for Return the name of the compression library for Blosc SphinxTranslator.dispatch_departure calling depart_Text for Return the name of the compression library for Blosc SphinxTranslator.dispatch_visit calling visit_literal for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_literal for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for buffer. SphinxTranslator.dispatch_departure calling depart_Text for buffer. SphinxTranslator.dispatch_departure calling depart_paragraph for Return the name of the compression library for Blosc bytesobj buffer. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_visit calling visit_field for Parametersbytesobj : str / bytesThe compressed buffer. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for bytesobj : str / bytesThe compressed buffer. SphinxTranslator.dispatch_visit calling visit_paragraph for bytesobj : str / bytes SphinxTranslator.dispatch_visit calling visit_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_Text for bytesobj SphinxTranslator.dispatch_departure calling depart_strong for bytesobj SphinxTranslator.dispatch_visit calling visit_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_Text for : str / bytes SphinxTranslator.dispatch_departure calling depart_paragraph for bytesobj : str / bytes SphinxTranslator.dispatch_visit calling visit_block_quote for The compressed buffer. SphinxTranslator.dispatch_visit calling visit_paragraph for The compressed buffer. SphinxTranslator.dispatch_visit calling visit_Text for The compressed buffer. SphinxTranslator.dispatch_departure calling depart_Text for The compressed buffer. SphinxTranslator.dispatch_departure calling depart_paragraph for The compressed buffer. SphinxTranslator.dispatch_departure calling depart_block_quote for The compressed buffer. SphinxTranslator.dispatch_departure calling depart_field_body for bytesobj : str / bytesThe compressed buffer. SphinxTranslator.dispatch_departure calling depart_field for Parametersbytesobj : str / bytesThe compressed buffer. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : strThe name of the compression library. SphinxTranslator.dispatch_visit calling visit_paragraph for out : str SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : str SphinxTranslator.dispatch_departure calling depart_Text for : str SphinxTranslator.dispatch_departure calling depart_paragraph for out : str SphinxTranslator.dispatch_visit calling visit_block_quote for The name of the compression library. SphinxTranslator.dispatch_visit calling visit_paragraph for The name of the compression library. SphinxTranslator.dispatch_visit calling visit_Text for The name of the compression library. SphinxTranslator.dispatch_departure calling depart_Text for The name of the compression library. SphinxTranslator.dispatch_departure calling depart_paragraph for The name of the compression library. SphinxTranslator.dispatch_departure calling depart_block_quote for The name of the compression library. SphinxTranslator.dispatch_departure calling depart_field_body for out : strThe name of the compression library. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_departure calling depart_desc_content for Return the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_departure calling depart_desc for blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library. SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512) SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.set_blocksizeblocksize SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for set_blocksize SphinxTranslator.dispatch_visit calling visit_Text for set_blocksize SphinxTranslator.dispatch_departure calling depart_Text for set_blocksize SphinxTranslator.dispatch_departure calling depart_desc_name for set_blocksize SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for blocksize SphinxTranslator.dispatch_visit calling visit_desc_parameter for blocksize SphinxTranslator.dispatch_visit calling visit_Text for blocksize SphinxTranslator.dispatch_departure calling depart_Text for blocksize SphinxTranslator.dispatch_departure calling depart_desc_parameter for blocksize SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for blocksize SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.set_blocksizeblocksize SphinxTranslator.dispatch_visit calling visit_desc_content for Force the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512) SphinxTranslator.dispatch_visit calling visit_paragraph for Force the use of a specific blocksize. If 0, an automatic blocksize will be used (the default). SphinxTranslator.dispatch_visit calling visit_Text for Force the use of a specific blocksize. If 0, an automatic blocksize will be used (the default). SphinxTranslator.dispatch_departure calling depart_Text for Force the use of a specific blocksize. If 0, an automatic blocksize will be used (the default). SphinxTranslator.dispatch_departure calling depart_paragraph for Force the use of a specific blocksize. If 0, an automatic blocksize will be used (the default). SphinxTranslator.dispatch_visit calling visit_rubric for Notes SphinxTranslator.dispatch_visit calling visit_Text for Notes SphinxTranslator.dispatch_departure calling depart_Text for Notes SphinxTranslator.dispatch_departure calling depart_rubric for Notes SphinxTranslator.dispatch_visit calling visit_paragraph for This is a low-level function and is recommened for expert users only. SphinxTranslator.dispatch_visit calling visit_Text for This is a low-level function and is recommened for expert users only. SphinxTranslator.dispatch_departure calling depart_Text for This is a low-level function and is recommened for expert users only. SphinxTranslator.dispatch_departure calling depart_paragraph for This is a low-level function and is recommened for expert users only. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> blosc.set_blocksize(512) SphinxTranslator.dispatch_departure calling depart_desc_content for Force the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512) SphinxTranslator.dispatch_departure calling depart_desc for blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512) SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2 SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.set_nthreadsnthreads SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for set_nthreads SphinxTranslator.dispatch_visit calling visit_Text for set_nthreads SphinxTranslator.dispatch_departure calling depart_Text for set_nthreads SphinxTranslator.dispatch_departure calling depart_desc_name for set_nthreads SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for nthreads SphinxTranslator.dispatch_visit calling visit_desc_parameter for nthreads SphinxTranslator.dispatch_visit calling visit_Text for nthreads SphinxTranslator.dispatch_departure calling depart_Text for nthreads SphinxTranslator.dispatch_departure calling depart_desc_parameter for nthreads SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for nthreads SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.set_nthreadsnthreads SphinxTranslator.dispatch_visit calling visit_desc_content for Set the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2 SphinxTranslator.dispatch_visit calling visit_paragraph for Set the number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_Text for Set the number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_Text for Set the number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_paragraph for Set the number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_field_list for Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_visit calling visit_field for Parametersnthreads : intThe number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_Text for Parameters SphinxTranslator.dispatch_departure calling depart_Text for Parameters SphinxTranslator.dispatch_departure calling depart_field_name for Parameters SphinxTranslator.dispatch_visit calling visit_field_body for nthreads : intThe number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_paragraph for nthreads : int SphinxTranslator.dispatch_visit calling visit_strong for nthreads SphinxTranslator.dispatch_visit calling visit_Text for nthreads SphinxTranslator.dispatch_departure calling depart_Text for nthreads SphinxTranslator.dispatch_departure calling depart_strong for nthreads SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for nthreads : int SphinxTranslator.dispatch_visit calling visit_block_quote for The number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_paragraph for The number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_Text for The number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_Text for The number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_paragraph for The number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_block_quote for The number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_field_body for nthreads : intThe number of threads to be used during Blosc operation. SphinxTranslator.dispatch_departure calling depart_field for Parametersnthreads : intThe number of threads to be used during Blosc operation. SphinxTranslator.dispatch_visit calling visit_field for Returnsout : intThe previous number of used threads. SphinxTranslator.dispatch_visit calling visit_field_name for Returns SphinxTranslator.dispatch_visit calling visit_Text for Returns SphinxTranslator.dispatch_departure calling depart_Text for Returns SphinxTranslator.dispatch_departure calling depart_field_name for Returns SphinxTranslator.dispatch_visit calling visit_field_body for out : intThe previous number of used threads. SphinxTranslator.dispatch_visit calling visit_paragraph for out : int SphinxTranslator.dispatch_visit calling visit_strong for out SphinxTranslator.dispatch_visit calling visit_Text for out SphinxTranslator.dispatch_departure calling depart_Text for out SphinxTranslator.dispatch_departure calling depart_strong for out SphinxTranslator.dispatch_visit calling visit_Text for : int SphinxTranslator.dispatch_departure calling depart_Text for : int SphinxTranslator.dispatch_departure calling depart_paragraph for out : int SphinxTranslator.dispatch_visit calling visit_block_quote for The previous number of used threads. SphinxTranslator.dispatch_visit calling visit_paragraph for The previous number of used threads. SphinxTranslator.dispatch_visit calling visit_Text for The previous number of used threads. SphinxTranslator.dispatch_departure calling depart_Text for The previous number of used threads. SphinxTranslator.dispatch_departure calling depart_paragraph for The previous number of used threads. SphinxTranslator.dispatch_departure calling depart_block_quote for The previous number of used threads. SphinxTranslator.dispatch_departure calling depart_field_body for out : intThe previous number of used threads. SphinxTranslator.dispatch_departure calling depart_field for Returnsout : intThe previous number of used threads. SphinxTranslator.dispatch_visit calling visit_field for RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_visit calling visit_field_name for Raises SphinxTranslator.dispatch_visit calling visit_Text for Raises SphinxTranslator.dispatch_departure calling depart_Text for Raises SphinxTranslator.dispatch_departure calling depart_field_name for Raises SphinxTranslator.dispatch_visit calling visit_field_body for ValueErrorIf nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_visit calling visit_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_strong for ValueError SphinxTranslator.dispatch_visit calling visit_Text for ValueError SphinxTranslator.dispatch_departure calling depart_Text for ValueError SphinxTranslator.dispatch_departure calling depart_strong for ValueError SphinxTranslator.dispatch_departure calling depart_paragraph for ValueError SphinxTranslator.dispatch_visit calling visit_block_quote for If nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_visit calling visit_paragraph for If nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_visit calling visit_Text for If nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_departure calling depart_Text for If nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_departure calling depart_paragraph for If nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_departure calling depart_block_quote for If nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_departure calling depart_field_body for ValueErrorIf nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_departure calling depart_field for RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_departure calling depart_field_list for Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use. SphinxTranslator.dispatch_visit calling visit_rubric for Notes SphinxTranslator.dispatch_visit calling visit_Text for Notes SphinxTranslator.dispatch_departure calling depart_Text for Notes SphinxTranslator.dispatch_departure calling depart_rubric for Notes SphinxTranslator.dispatch_visit calling visit_paragraph for The number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores. SphinxTranslator.dispatch_visit calling visit_Text for The number of threads for Blosc is the maximum number of cores detected on your machine (via SphinxTranslator.dispatch_departure calling depart_Text for The number of threads for Blosc is the maximum number of cores detected on your machine (via SphinxTranslator.dispatch_visit calling visit_literal for detect_number_of_cores SphinxTranslator.dispatch_visit calling visit_Text for detect_number_of_cores SphinxTranslator.dispatch_departure calling depart_Text for detect_number_of_cores SphinxTranslator.dispatch_departure calling depart_literal for detect_number_of_cores SphinxTranslator.dispatch_visit calling visit_Text for ). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores. SphinxTranslator.dispatch_departure calling depart_Text for ). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores. SphinxTranslator.dispatch_departure calling depart_paragraph for The number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_paragraph for Set the number of threads to 2 and then to 1: SphinxTranslator.dispatch_visit calling visit_Text for Set the number of threads to 2 and then to 1: SphinxTranslator.dispatch_departure calling depart_Text for Set the number of threads to 2 and then to 1: SphinxTranslator.dispatch_departure calling depart_paragraph for Set the number of threads to 2 and then to 1: SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2 SphinxTranslator.dispatch_departure calling depart_desc_content for Set the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2 SphinxTranslator.dispatch_departure calling depart_desc for blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2 SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True) SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.set_releasegilgitstate SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for set_releasegil SphinxTranslator.dispatch_visit calling visit_Text for set_releasegil SphinxTranslator.dispatch_departure calling depart_Text for set_releasegil SphinxTranslator.dispatch_departure calling depart_desc_name for set_releasegil SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for gitstate SphinxTranslator.dispatch_visit calling visit_desc_parameter for gitstate SphinxTranslator.dispatch_visit calling visit_Text for gitstate SphinxTranslator.dispatch_departure calling depart_Text for gitstate SphinxTranslator.dispatch_departure calling depart_desc_parameter for gitstate SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for gitstate SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.set_releasegilgitstate SphinxTranslator.dispatch_visit calling visit_desc_content for Sets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True) SphinxTranslator.dispatch_visit calling visit_paragraph for Sets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False. SphinxTranslator.dispatch_visit calling visit_Text for Sets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False. SphinxTranslator.dispatch_departure calling depart_Text for Sets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False. SphinxTranslator.dispatch_departure calling depart_paragraph for Sets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False. SphinxTranslator.dispatch_visit calling visit_rubric for Notes SphinxTranslator.dispatch_visit calling visit_Text for Notes SphinxTranslator.dispatch_departure calling depart_Text for Notes SphinxTranslator.dispatch_departure calling depart_rubric for Notes SphinxTranslator.dispatch_visit calling visit_paragraph for Designed to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes. SphinxTranslator.dispatch_visit calling visit_Text for Designed to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes. SphinxTranslator.dispatch_departure calling depart_Text for Designed to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes. SphinxTranslator.dispatch_departure calling depart_paragraph for Designed to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes. SphinxTranslator.dispatch_visit calling visit_rubric for Examples SphinxTranslator.dispatch_visit calling visit_Text for Examples SphinxTranslator.dispatch_departure calling depart_Text for Examples SphinxTranslator.dispatch_departure calling depart_rubric for Examples SphinxTranslator.dispatch_visit calling visit_doctest_block for >>> oldReleaseState = blosc.set_releasegil(True) SphinxTranslator.dispatch_departure calling depart_desc_content for Sets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True) SphinxTranslator.dispatch_departure calling depart_desc for blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True) SphinxTranslator.dispatch_visit calling visit_index for SphinxTranslator.dispatch_visit calling visit_desc for blosc.print_versionsPrint all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_visit calling visit_desc_signature for blosc.print_versions SphinxTranslator.dispatch_visit calling visit_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_Text for blosc. SphinxTranslator.dispatch_departure calling depart_Text for blosc. SphinxTranslator.dispatch_departure calling depart_desc_addname for blosc. SphinxTranslator.dispatch_visit calling visit_desc_name for print_versions SphinxTranslator.dispatch_visit calling visit_Text for print_versions SphinxTranslator.dispatch_departure calling depart_Text for print_versions SphinxTranslator.dispatch_departure calling depart_desc_name for print_versions SphinxTranslator.dispatch_visit calling visit_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_parameterlist for SphinxTranslator.dispatch_departure calling depart_desc_signature for blosc.print_versions SphinxTranslator.dispatch_visit calling visit_desc_content for Print all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_visit calling visit_paragraph for Print all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_visit calling visit_Text for Print all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_departure calling depart_Text for Print all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_departure calling depart_paragraph for Print all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_departure calling depart_desc_content for Print all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_departure calling depart_desc for blosc.print_versionsPrint all the versions of software that python-blosc relies on. SphinxTranslator.dispatch_departure calling depart_section for
Utilitiesblosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version.blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names.blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system.blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>>blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library.blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512)blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True)blosc.print_versionsPrint all the versions of software that python-blosc relies on.
SphinxTranslator.dispatch_departure calling depart_section for
Library Reference
First level variables__version__The version of the blosc package.blosclib_versionThe version of the Blosc C library.clib_versionsA map for the versions of the compression libraries included in C library.cnamesThe list of compressors included in C library.cname2clibA map between compressor names and its libraries (or formats).ncoresThe number of cores detected.
Public functionsblosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) Trueblosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans Trueblosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray Trueblosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize Trueblosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize Trueblosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True
Utilitiesblosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version.blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names.blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system.blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>>blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library.blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512)blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True)blosc.print_versionsPrint all the versions of software that python-blosc relies on.
SphinxTranslator.dispatch_departure calling depart_document for
Library Reference
First level variables__version__The version of the blosc package.blosclib_versionThe version of the Blosc C library.clib_versionsA map for the versions of the compression libraries included in C library.cnamesThe list of compressors included in C library.cname2clibA map between compressor names and its libraries (or formats).ncoresThe number of cores detected.
Public functionsblosc.compressbytesobj[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress bytesobj, with a given type size.Parametersbytesobj : bytes-like object (supporting the buffer interface)The data to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf bytesobj doesn’t support the buffer interface.ValueErrorIf bytesobj is too long. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not a valid codec.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> len(c_bytesobj) < len(a_bytesobj) Trueblosc.compress_ptraddress, items[, typesize=8, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Compress the data at address with given items and typesize.Parametersaddress : int or longthe pointer to the data to be compresseditems : intThe number of items (of typesize) to be compressed.typesize : intThe data type size.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe compressed data in form of a Python str / bytes object.RaisesTypeErrorIf address is not of type int or long.ValueErrorIf items * typesize is larger than the maximum allowed buffer size. If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> d = blosc.decompress(c) >>> np_ans = numpy.fromstring(d, dtype=np_array.dtype) >>> (np_array == np_ans).all() True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> a = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(a), items, typesize) >>> d = blosc.decompress(c) >>> import struct >>> ans = [struct.unpack('d', d[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans Trueblosc.decompressbytesobjDecompresses a bytesobj compressed object.Parametersbytesobj : str / bytesThe data to be decompressed.as_bytearray : bool, optionalIf this flag is True then the return type will be a bytearray object instead of a bytesobject.Returnsout : str / bytes or bytearrayThe decompressed data in form of a Python str / bytes object. If as_bytearray is True then this will be a bytearray object, otherwise this will be a str/ bytes object.RaisesTypeErrorIf bytesobj is not of type bytes or string.Examples>>> import array >>> a = array.array('i', range(1000*1000)) >>> a_bytesobj = a.tostring() >>> c_bytesobj = blosc.compress(a_bytesobj, typesize=4) >>> a_bytesobj2 = blosc.decompress(c_bytesobj) >>> a_bytesobj == a_bytesobj2 True >>> b"" == blosc.decompress(blosc.compress(b"", 1)) True >>> b"1"*7 == blosc.decompress(blosc.compress(b"1"*7, 8)) True >>> type(blosc.decompress(blosc.compress(b"1"*7, 8), ... as_bytearray=True)) is bytearray Trueblosc.decompress_ptrbytesobjaddressDecompresses a bytesobj compressed object into the memory at address.Parametersbytesobj : str / bytesThe data to be decompressed.address : int or longthe pointer to the data to be compressedReturnsnbytes : intthe number of bytes written to the bufferRaisesTypeErrorIf bytesobj is not of type bytes or string. If address is not of type int or long.NotesThis function can be used anywhere that a memory address is available in Python. For example the Numpy “__array_interface__[‘data’][0]” construct, or when using the ctypes modules.Importantly, the user is responsible for making sure that the memory address is valid and that the memory pointed to is contiguous and can be written to. Passing a non-valid address has a high likelihood of crashing the interpreter by segfault.Examples>>> import numpy >>> items = 7 >>> np_array = numpy.arange(items) >>> c = blosc.compress_ptr(np_array.__array_interface__['data'][0], items, np_array.dtype.itemsize) >>> np_ans = numpy.empty(items, dtype=np_array.dtype) >>> nbytes = blosc.decompress_ptr(c, np_ans.__array_interface__['data'][0]) >>> (np_array == np_ans).all() True >>> nbytes == items * np_array.dtype.itemsize True>>> import ctypes >>> typesize = 8 >>> data = [float(i) for i in range(items)] >>> Array = ctypes.c_double * items >>> in_array = Array(*data) >>> c = blosc.compress_ptr(ctypes.addressof(in_array), items, typesize) >>> out_array = ctypes.create_string_buffer(items*typesize) >>> nbytes = blosc.decompress_ptr(c, ctypes.addressof(out_array)) >>> import struct >>> ans = [struct.unpack('d', out_array[i:i+typesize])[0] for i in range(0, items*typesize, typesize)] >>> data == ans True >>> nbytes == items * typesize Trueblosc.pack_arrayarray[, clevel=9, shuffle=blosc.SHUFFLE, cname='blosclz']]Pack (compress) a NumPy array.Parametersarray : ndarrayThe NumPy array to be packed.clevel : int (optional)The compression level from 0 (no compression) to 9 (maximum compression). The default is 9.shuffle : int (optional)The shuffle filter to be activated. Allowed values are blosc.NOSHUFFLE, blosc.SHUFFLE and blosc.BITSHUFFLE. The default is blosc.SHUFFLE.cname : string (optional)The name of the compressor used internally in Blosc. It can be any of the supported by Blosc (‘blosclz’, ‘lz4’, ‘lz4hc’, ‘snappy’, ‘zlib’, ‘zstd’ and maybe others too). The default is ‘blosclz’.Returnsout : str / bytesThe packed array in form of a Python str / bytes object.RaisesTypeErrorIf array does not quack like a numpy ndarray.ValueErrorIf array.itemsize * array.size is larger than the maximum allowedbuffer size.If typesize is not within the allowed range. If clevel is not within the allowed range. If cname is not within the supported compressors.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize Trueblosc.unpack_arraypacked_arrayUnpack (decompress) a packed NumPy array.Parameterspacked_array : str / bytesThe packed array to be decompressed.Returnsout : ndarrayThe decompressed data in form of a NumPy array.RaisesTypeErrorIf packed_array is not of type bytes or string.Examples>>> import numpy >>> a = numpy.arange(1e6) >>> parray = blosc.pack_array(a) >>> len(parray) < a.size*a.itemsize True >>> a2 = blosc.unpack_array(parray) >>> numpy.alltrue(a == a2) True
Utilitiesblosc.clib_infocnameReturn info for compression libraries in C library.Parameterscname : strThe compressor name.Returnsout : tupleThe associated library name and version.blosc.compressor_listReturns a list of compressors available in C library.ParametersNoneReturnsout : listThe list of names.blosc.detect_number_of_coresDetect the number of cores in this system.Returnsout : intThe number of cores in this system.blosc.free_resourcesFree possible memory temporaries and thread resources.Returnsout : NoneNotesBlosc maintain a pool of threads waiting for work as well as some temporary space. You can use this function to release these resources when you are not going to use Blosc for a long while.Examples>>> blosc.free_resources() >>>blosc.get_clibbytesobjReturn the name of the compression library for Blosc bytesobj buffer.Parametersbytesobj : str / bytesThe compressed buffer.Returnsout : strThe name of the compression library.blosc.set_blocksizeblocksizeForce the use of a specific blocksize. If 0, an automatic blocksize will be used (the default).NotesThis is a low-level function and is recommened for expert users only.Examples>>> blosc.set_blocksize(512)blosc.set_nthreadsnthreadsSet the number of threads to be used during Blosc operation.Parametersnthreads : intThe number of threads to be used during Blosc operation.Returnsout : intThe previous number of used threads.RaisesValueErrorIf nthreads is larger that the maximum number of threads blosc can use.NotesThe number of threads for Blosc is the maximum number of cores detected on your machine (via detect_number_of_cores). In some cases Blosc gets better results if you set the number of threads to a value slightly below than your number of cores.ExamplesSet the number of threads to 2 and then to 1:>>> oldn = blosc.set_nthreads(2) >>> blosc.set_nthreads(1) 2blosc.set_releasegilgitstateSets a boolean on whether to release the Python global inter-lock (GIL) during c-blosc compress and decompress operations or not. This defaults to False.NotesDesigned to be used with larger chunk sizes and a ThreadPool. There is a small performance penalty with releasing the GIL that will more harshly penalize small block sizes.Examples>>> oldReleaseState = blosc.set_releasegil(True)blosc.print_versionsPrint all the versions of software that python-blosc relies on.
SphinxTranslator.dispatch_visit calling visit_document for Tutorials SphinxTranslator.dispatch_visit calling visit_title for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_title for Tutorials SphinxTranslator.dispatch_departure calling depart_document for Tutorials SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for Library Reference SphinxTranslator.dispatch_visit calling visit_title for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_title for Library Reference SphinxTranslator.dispatch_departure calling depart_document for Library Reference SphinxTranslator.dispatch_visit calling visit_document for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_bullet_list for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_list_item for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Library Reference SphinxTranslator.dispatch_visit calling visit_reference for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_reference for Library Reference SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Library Reference SphinxTranslator.dispatch_visit calling visit_bullet_list for First level variablesPublic functionsUtilities SphinxTranslator.dispatch_visit calling visit_list_item for First level variables SphinxTranslator.dispatch_visit calling visit_compact_paragraph for First level variables SphinxTranslator.dispatch_visit calling visit_reference for First level variables SphinxTranslator.dispatch_visit calling visit_Text for First level variables SphinxTranslator.dispatch_departure calling depart_Text for First level variables SphinxTranslator.dispatch_departure calling depart_reference for First level variables SphinxTranslator.dispatch_departure calling depart_compact_paragraph for First level variables SphinxTranslator.dispatch_departure calling depart_list_item for First level variables SphinxTranslator.dispatch_visit calling visit_list_item for Public functions SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Public functions SphinxTranslator.dispatch_visit calling visit_reference for Public functions SphinxTranslator.dispatch_visit calling visit_Text for Public functions SphinxTranslator.dispatch_departure calling depart_Text for Public functions SphinxTranslator.dispatch_departure calling depart_reference for Public functions SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Public functions SphinxTranslator.dispatch_departure calling depart_list_item for Public functions SphinxTranslator.dispatch_visit calling visit_list_item for Utilities SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Utilities SphinxTranslator.dispatch_visit calling visit_reference for Utilities SphinxTranslator.dispatch_visit calling visit_Text for Utilities SphinxTranslator.dispatch_departure calling depart_Text for Utilities SphinxTranslator.dispatch_departure calling depart_reference for Utilities SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Utilities SphinxTranslator.dispatch_departure calling depart_list_item for Utilities SphinxTranslator.dispatch_departure calling depart_bullet_list for First level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_list_item for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_bullet_list for Library ReferenceFirst level variablesPublic functionsUtilities SphinxTranslator.dispatch_departure calling depart_document for Library ReferenceFirst level variablesPublic functionsUtilities [app] emitting event: 'html-page-context'('reference', 'page.html', {'embedded': False, 'project': 'python-blosc', 'release': '1.5.0', 'versi writing output... [100%] tutorial [app] emitting event: 'doctree-resolved'(>, 'tutorial') SphinxTranslator.dispatch_visit calling visit_document for Tutorials SphinxTranslator.dispatch_visit calling visit_title for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_title for Tutorials SphinxTranslator.dispatch_departure calling depart_document for Tutorials SphinxTranslator.dispatch_visit calling visit_document for
TutorialsUsing python-blosc (or just blosc, because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the zlib module included in the standard Python library.Here are some examples on how to use it. For the full documentation, please refer to the Library Reference section.Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz.
Compressing and decompressing with <literal>blosc</literal>Let’s start creating a NumPy array with 80 MB full of data:>>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytesand let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities):>>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlibbut Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default):>>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlibIt is important to note that this is quite more than the speed of a memcpy() in this machine:>>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/swhich means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises.Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib:>>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlibThe reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs.Now, let’s have a look at the compression ratios:>>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratioHere we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string:>>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib:>>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLibHere it is a plot with the different compression ratios achieved:The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data.How about decompression?:>>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlibHere we see a couple of things:All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression).The next plot summarizes the speed benchmarks above:These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation).Finally, here it is the way to discover all the internal codecs inside your Blosc package:>>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library.
Using different filtersIn the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets.Here it is an example using the SHUFFLE filter:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533Here there is another example using BITSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffleYou can also deactivate filters completely with NOSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original bufferSo you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one.
Supporting the buffer interfacepython-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray:>>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz'Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode.
Packaging NumPy arraysWant to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way:>>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/sAlthough this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time.
Compressing from a data pointerFor avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action:>>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() TrueAs you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size).On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization.blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as:>>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True)
Fine-tuning compression parametersThere are a number of parameters that affect the de/compression bandwidth achieved by blosc:The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py.
Links to external discussions on <literal>blosc</literal> optimizationSynthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles
SphinxTranslator.dispatch_visit calling visit_section for
TutorialsUsing python-blosc (or just blosc, because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the zlib module included in the standard Python library.Here are some examples on how to use it. For the full documentation, please refer to the Library Reference section.Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz.
Compressing and decompressing with <literal>blosc</literal>Let’s start creating a NumPy array with 80 MB full of data:>>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytesand let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities):>>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlibbut Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default):>>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlibIt is important to note that this is quite more than the speed of a memcpy() in this machine:>>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/swhich means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises.Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib:>>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlibThe reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs.Now, let’s have a look at the compression ratios:>>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratioHere we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string:>>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib:>>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLibHere it is a plot with the different compression ratios achieved:The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data.How about decompression?:>>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlibHere we see a couple of things:All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression).The next plot summarizes the speed benchmarks above:These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation).Finally, here it is the way to discover all the internal codecs inside your Blosc package:>>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library.
Using different filtersIn the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets.Here it is an example using the SHUFFLE filter:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533Here there is another example using BITSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffleYou can also deactivate filters completely with NOSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original bufferSo you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one.
Supporting the buffer interfacepython-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray:>>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz'Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode.
Packaging NumPy arraysWant to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way:>>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/sAlthough this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time.
Compressing from a data pointerFor avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action:>>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() TrueAs you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size).On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization.blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as:>>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True)
Fine-tuning compression parametersThere are a number of parameters that affect the de/compression bandwidth achieved by blosc:The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py.
Links to external discussions on <literal>blosc</literal> optimizationSynthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles
SphinxTranslator.dispatch_visit calling visit_title for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_title for Tutorials SphinxTranslator.dispatch_visit calling visit_paragraph for Using python-blosc (or just blosc, because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the zlib module included in the standard Python library. SphinxTranslator.dispatch_visit calling visit_Text for Using SphinxTranslator.dispatch_departure calling depart_Text for Using SphinxTranslator.dispatch_visit calling visit_literal for python-blosc SphinxTranslator.dispatch_visit calling visit_Text for python-blosc SphinxTranslator.dispatch_departure calling depart_Text for python-blosc SphinxTranslator.dispatch_departure calling depart_literal for python-blosc SphinxTranslator.dispatch_visit calling visit_Text for (or just SphinxTranslator.dispatch_departure calling depart_Text for (or just SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for , because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the SphinxTranslator.dispatch_departure calling depart_Text for , because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the SphinxTranslator.dispatch_visit calling visit_literal for zlib SphinxTranslator.dispatch_visit calling visit_Text for zlib SphinxTranslator.dispatch_departure calling depart_Text for zlib SphinxTranslator.dispatch_departure calling depart_literal for zlib SphinxTranslator.dispatch_visit calling visit_Text for module included in the standard Python library. SphinxTranslator.dispatch_departure calling depart_Text for module included in the standard Python library. SphinxTranslator.dispatch_departure calling depart_paragraph for Using python-blosc (or just blosc, because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the zlib module included in the standard Python library. SphinxTranslator.dispatch_visit calling visit_paragraph for Here are some examples on how to use it. For the full documentation, please refer to the Library Reference section. SphinxTranslator.dispatch_visit calling visit_Text for Here are some examples on how to use it. For the full documentation, please refer to the SphinxTranslator.dispatch_departure calling depart_Text for Here are some examples on how to use it. For the full documentation, please refer to the SphinxTranslator.dispatch_visit calling visit_reference for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_reference for Library Reference SphinxTranslator.dispatch_visit calling visit_target for SphinxTranslator.dispatch_departure calling depart_target for SphinxTranslator.dispatch_visit calling visit_Text for section. SphinxTranslator.dispatch_departure calling depart_Text for section. SphinxTranslator.dispatch_departure calling depart_paragraph for Here are some examples on how to use it. For the full documentation, please refer to the Library Reference section. SphinxTranslator.dispatch_visit calling visit_paragraph for Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz. SphinxTranslator.dispatch_visit calling visit_Text for Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz. SphinxTranslator.dispatch_departure calling depart_Text for Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz. SphinxTranslator.dispatch_departure calling depart_paragraph for Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz. SphinxTranslator.dispatch_visit calling visit_section for
Compressing and decompressing with <literal>blosc</literal>Let’s start creating a NumPy array with 80 MB full of data:>>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytesand let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities):>>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlibbut Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default):>>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlibIt is important to note that this is quite more than the speed of a memcpy() in this machine:>>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/swhich means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises.Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib:>>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlibThe reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs.Now, let’s have a look at the compression ratios:>>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratioHere we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string:>>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib:>>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLibHere it is a plot with the different compression ratios achieved:The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data.How about decompression?:>>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlibHere we see a couple of things:All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression).The next plot summarizes the speed benchmarks above:These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation).Finally, here it is the way to discover all the internal codecs inside your Blosc package:>>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library.
SphinxTranslator.dispatch_visit calling visit_title for Compressing and decompressing with <literal>blosc</literal> SphinxTranslator.dispatch_visit calling visit_Text for Compressing and decompressing with SphinxTranslator.dispatch_departure calling depart_Text for Compressing and decompressing with SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_departure calling depart_title for Compressing and decompressing with <literal>blosc</literal> SphinxTranslator.dispatch_visit calling visit_paragraph for Let’s start creating a NumPy array with 80 MB full of data: SphinxTranslator.dispatch_visit calling visit_Text for Let’s start creating a NumPy array with 80 MB full of data: SphinxTranslator.dispatch_departure calling depart_Text for Let’s start creating a NumPy array with 80 MB full of data: SphinxTranslator.dispatch_departure calling depart_paragraph for Let’s start creating a NumPy array with 80 MB full of data: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytes SphinxTranslator.dispatch_visit calling visit_paragraph for and let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities): SphinxTranslator.dispatch_visit calling visit_Text for and let’s compare Blosc operation with SphinxTranslator.dispatch_departure calling depart_Text for and let’s compare Blosc operation with SphinxTranslator.dispatch_visit calling visit_literal for zlib SphinxTranslator.dispatch_visit calling visit_Text for zlib SphinxTranslator.dispatch_departure calling depart_Text for zlib SphinxTranslator.dispatch_departure calling depart_literal for zlib SphinxTranslator.dispatch_visit calling visit_Text for (please note that we are using IPython for leveraging its timing capabilities): SphinxTranslator.dispatch_departure calling depart_Text for (please note that we are using IPython for leveraging its timing capabilities): SphinxTranslator.dispatch_departure calling depart_paragraph for and let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities): SphinxTranslator.dispatch_visit calling visit_literal_block for >>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlib SphinxTranslator.dispatch_visit calling visit_paragraph for but Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default): SphinxTranslator.dispatch_visit calling visit_Text for but Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default): SphinxTranslator.dispatch_departure calling depart_Text for but Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default): SphinxTranslator.dispatch_departure calling depart_paragraph for but Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default): SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlib SphinxTranslator.dispatch_visit calling visit_paragraph for It is important to note that this is quite more than the speed of a memcpy() in this machine: SphinxTranslator.dispatch_visit calling visit_Text for It is important to note that this is quite more than the speed of a memcpy() in this machine: SphinxTranslator.dispatch_departure calling depart_Text for It is important to note that this is quite more than the speed of a memcpy() in this machine: SphinxTranslator.dispatch_departure calling depart_paragraph for It is important to note that this is quite more than the speed of a memcpy() in this machine: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/s SphinxTranslator.dispatch_visit calling visit_paragraph for which means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises. SphinxTranslator.dispatch_visit calling visit_Text for which means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises. SphinxTranslator.dispatch_departure calling depart_Text for which means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises. SphinxTranslator.dispatch_departure calling depart_paragraph for which means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises. SphinxTranslator.dispatch_visit calling visit_paragraph for Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib: SphinxTranslator.dispatch_visit calling visit_Text for Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib: SphinxTranslator.dispatch_departure calling depart_Text for Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib: SphinxTranslator.dispatch_departure calling depart_paragraph for Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlib SphinxTranslator.dispatch_visit calling visit_paragraph for The reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs. SphinxTranslator.dispatch_visit calling visit_Text for The reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs. SphinxTranslator.dispatch_departure calling depart_Text for The reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs. SphinxTranslator.dispatch_departure calling depart_paragraph for The reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs. SphinxTranslator.dispatch_visit calling visit_paragraph for Now, let’s have a look at the compression ratios: SphinxTranslator.dispatch_visit calling visit_Text for Now, let’s have a look at the compression ratios: SphinxTranslator.dispatch_departure calling depart_Text for Now, let’s have a look at the compression ratios: SphinxTranslator.dispatch_departure calling depart_paragraph for Now, let’s have a look at the compression ratios: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratio SphinxTranslator.dispatch_visit calling visit_paragraph for Here we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string: SphinxTranslator.dispatch_visit calling visit_Text for Here we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string: SphinxTranslator.dispatch_departure calling depart_Text for Here we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string: SphinxTranslator.dispatch_departure calling depart_paragraph for Here we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996 SphinxTranslator.dispatch_visit calling visit_paragraph for That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib: SphinxTranslator.dispatch_visit calling visit_Text for That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib: SphinxTranslator.dispatch_departure calling depart_Text for That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib: SphinxTranslator.dispatch_departure calling depart_paragraph for That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLib SphinxTranslator.dispatch_visit calling visit_paragraph for Here it is a plot with the different compression ratios achieved: SphinxTranslator.dispatch_visit calling visit_Text for Here it is a plot with the different compression ratios achieved: SphinxTranslator.dispatch_departure calling depart_Text for Here it is a plot with the different compression ratios achieved: SphinxTranslator.dispatch_departure calling depart_paragraph for Here it is a plot with the different compression ratios achieved: SphinxTranslator.dispatch_visit calling visit_reference for SphinxTranslator.dispatch_visit calling visit_image for SphinxTranslator.dispatch_departure calling depart_image for SphinxTranslator.dispatch_departure calling depart_reference for SphinxTranslator.dispatch_visit calling visit_paragraph for The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data. SphinxTranslator.dispatch_visit calling visit_Text for The explanation for this apparently shocking result is that Blosc uses filters ( SphinxTranslator.dispatch_departure calling depart_Text for The explanation for this apparently shocking result is that Blosc uses filters ( SphinxTranslator.dispatch_visit calling visit_literal for SHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for SHUFFLE SphinxTranslator.dispatch_departure calling depart_Text for SHUFFLE SphinxTranslator.dispatch_departure calling depart_literal for SHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for BITSHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for BITSHUFFLE SphinxTranslator.dispatch_departure calling depart_Text for BITSHUFFLE SphinxTranslator.dispatch_departure calling depart_literal for BITSHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data. SphinxTranslator.dispatch_departure calling depart_Text for currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data. SphinxTranslator.dispatch_departure calling depart_paragraph for The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data. SphinxTranslator.dispatch_visit calling visit_paragraph for How about decompression?: SphinxTranslator.dispatch_visit calling visit_Text for How about decompression?: SphinxTranslator.dispatch_departure calling depart_Text for How about decompression?: SphinxTranslator.dispatch_departure calling depart_paragraph for How about decompression?: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlib SphinxTranslator.dispatch_visit calling visit_paragraph for Here we see a couple of things: SphinxTranslator.dispatch_visit calling visit_Text for Here we see a couple of things: SphinxTranslator.dispatch_departure calling depart_Text for Here we see a couple of things: SphinxTranslator.dispatch_departure calling depart_paragraph for Here we see a couple of things: SphinxTranslator.dispatch_visit calling visit_bullet_list for All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_visit calling visit_list_item for All the internal codecs in Blosc are way faster than naked ZLib SphinxTranslator.dispatch_visit calling visit_paragraph for All the internal codecs in Blosc are way faster than naked ZLib SphinxTranslator.dispatch_visit calling visit_Text for All the internal codecs in Blosc are way faster than naked ZLib SphinxTranslator.dispatch_departure calling depart_Text for All the internal codecs in Blosc are way faster than naked ZLib SphinxTranslator.dispatch_departure calling depart_paragraph for All the internal codecs in Blosc are way faster than naked ZLib SphinxTranslator.dispatch_departure calling depart_list_item for All the internal codecs in Blosc are way faster than naked ZLib SphinxTranslator.dispatch_visit calling visit_list_item for The fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_visit calling visit_paragraph for The fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_visit calling visit_Text for The fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_departure calling depart_Text for The fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_departure calling depart_paragraph for The fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_departure calling depart_list_item for The fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_departure calling depart_bullet_list for All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression). SphinxTranslator.dispatch_visit calling visit_paragraph for The next plot summarizes the speed benchmarks above: SphinxTranslator.dispatch_visit calling visit_Text for The next plot summarizes the speed benchmarks above: SphinxTranslator.dispatch_departure calling depart_Text for The next plot summarizes the speed benchmarks above: SphinxTranslator.dispatch_departure calling depart_paragraph for The next plot summarizes the speed benchmarks above: SphinxTranslator.dispatch_visit calling visit_reference for SphinxTranslator.dispatch_visit calling visit_image for SphinxTranslator.dispatch_departure calling depart_image for SphinxTranslator.dispatch_departure calling depart_reference for SphinxTranslator.dispatch_visit calling visit_paragraph for These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation). SphinxTranslator.dispatch_visit calling visit_Text for These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation). SphinxTranslator.dispatch_departure calling depart_Text for These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation). SphinxTranslator.dispatch_departure calling depart_paragraph for These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation). SphinxTranslator.dispatch_visit calling visit_paragraph for Finally, here it is the way to discover all the internal codecs inside your Blosc package: SphinxTranslator.dispatch_visit calling visit_Text for Finally, here it is the way to discover all the internal codecs inside your Blosc package: SphinxTranslator.dispatch_departure calling depart_Text for Finally, here it is the way to discover all the internal codecs inside your Blosc package: SphinxTranslator.dispatch_departure calling depart_paragraph for Finally, here it is the way to discover all the internal codecs inside your Blosc package: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd'] SphinxTranslator.dispatch_visit calling visit_paragraph for Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library. SphinxTranslator.dispatch_visit calling visit_emphasis for Note SphinxTranslator.dispatch_visit calling visit_Text for Note SphinxTranslator.dispatch_departure calling depart_Text for Note SphinxTranslator.dispatch_departure calling depart_emphasis for Note SphinxTranslator.dispatch_visit calling visit_Text for : the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library. SphinxTranslator.dispatch_departure calling depart_Text for : the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library. SphinxTranslator.dispatch_departure calling depart_paragraph for Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library. SphinxTranslator.dispatch_departure calling depart_section for
Compressing and decompressing with <literal>blosc</literal>Let’s start creating a NumPy array with 80 MB full of data:>>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytesand let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities):>>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlibbut Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default):>>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlibIt is important to note that this is quite more than the speed of a memcpy() in this machine:>>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/swhich means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises.Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib:>>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlibThe reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs.Now, let’s have a look at the compression ratios:>>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratioHere we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string:>>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib:>>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLibHere it is a plot with the different compression ratios achieved:The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data.How about decompression?:>>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlibHere we see a couple of things:All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression).The next plot summarizes the speed benchmarks above:These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation).Finally, here it is the way to discover all the internal codecs inside your Blosc package:>>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library.
SphinxTranslator.dispatch_visit calling visit_section for
Using different filtersIn the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets.Here it is an example using the SHUFFLE filter:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533Here there is another example using BITSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffleYou can also deactivate filters completely with NOSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original bufferSo you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one.
SphinxTranslator.dispatch_visit calling visit_title for Using different filters SphinxTranslator.dispatch_visit calling visit_Text for Using different filters SphinxTranslator.dispatch_departure calling depart_Text for Using different filters SphinxTranslator.dispatch_departure calling depart_title for Using different filters SphinxTranslator.dispatch_visit calling visit_paragraph for In the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets. SphinxTranslator.dispatch_visit calling visit_Text for In the same way that you can use different codecs inside Blosc, you can use different filters too (currently SphinxTranslator.dispatch_departure calling depart_Text for In the same way that you can use different codecs inside Blosc, you can use different filters too (currently SphinxTranslator.dispatch_visit calling visit_literal for SHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for SHUFFLE SphinxTranslator.dispatch_departure calling depart_Text for SHUFFLE SphinxTranslator.dispatch_departure calling depart_literal for SHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for BITSHUFLE SphinxTranslator.dispatch_visit calling visit_Text for BITSHUFLE SphinxTranslator.dispatch_departure calling depart_Text for BITSHUFLE SphinxTranslator.dispatch_departure calling depart_literal for BITSHUFLE SphinxTranslator.dispatch_visit calling visit_Text for ). These allow the integrated compressors to compress more efficiently or not, depending on your datasets. SphinxTranslator.dispatch_departure calling depart_Text for ). These allow the integrated compressors to compress more efficiently or not, depending on your datasets. SphinxTranslator.dispatch_departure calling depart_paragraph for In the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets. SphinxTranslator.dispatch_visit calling visit_paragraph for Here it is an example using the SHUFFLE filter: SphinxTranslator.dispatch_visit calling visit_Text for Here it is an example using the SphinxTranslator.dispatch_departure calling depart_Text for Here it is an example using the SphinxTranslator.dispatch_visit calling visit_literal for SHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for SHUFFLE SphinxTranslator.dispatch_departure calling depart_Text for SHUFFLE SphinxTranslator.dispatch_departure calling depart_literal for SHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for filter: SphinxTranslator.dispatch_departure calling depart_Text for filter: SphinxTranslator.dispatch_departure calling depart_paragraph for Here it is an example using the SHUFFLE filter: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533 SphinxTranslator.dispatch_visit calling visit_paragraph for Here there is another example using BITSHUFFLE: SphinxTranslator.dispatch_visit calling visit_Text for Here there is another example using SphinxTranslator.dispatch_departure calling depart_Text for Here there is another example using SphinxTranslator.dispatch_visit calling visit_literal for BITSHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for BITSHUFFLE SphinxTranslator.dispatch_departure calling depart_Text for BITSHUFFLE SphinxTranslator.dispatch_departure calling depart_literal for BITSHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for : SphinxTranslator.dispatch_departure calling depart_Text for : SphinxTranslator.dispatch_departure calling depart_paragraph for Here there is another example using BITSHUFFLE: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffle SphinxTranslator.dispatch_visit calling visit_paragraph for You can also deactivate filters completely with NOSHUFFLE: SphinxTranslator.dispatch_visit calling visit_Text for You can also deactivate filters completely with SphinxTranslator.dispatch_departure calling depart_Text for You can also deactivate filters completely with SphinxTranslator.dispatch_visit calling visit_literal for NOSHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for NOSHUFFLE SphinxTranslator.dispatch_departure calling depart_Text for NOSHUFFLE SphinxTranslator.dispatch_departure calling depart_literal for NOSHUFFLE SphinxTranslator.dispatch_visit calling visit_Text for : SphinxTranslator.dispatch_departure calling depart_Text for : SphinxTranslator.dispatch_departure calling depart_paragraph for You can also deactivate filters completely with NOSHUFFLE: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original buffer SphinxTranslator.dispatch_visit calling visit_paragraph for So you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one. SphinxTranslator.dispatch_visit calling visit_Text for So you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one. SphinxTranslator.dispatch_departure calling depart_Text for So you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one. SphinxTranslator.dispatch_departure calling depart_paragraph for So you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one. SphinxTranslator.dispatch_departure calling depart_section for
Using different filtersIn the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets.Here it is an example using the SHUFFLE filter:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533Here there is another example using BITSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffleYou can also deactivate filters completely with NOSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original bufferSo you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one.
SphinxTranslator.dispatch_visit calling visit_section for
Supporting the buffer interfacepython-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray:>>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz'Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode.
SphinxTranslator.dispatch_visit calling visit_title for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_Text for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_Text for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_title for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_paragraph for python-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray: SphinxTranslator.dispatch_visit calling visit_Text for python-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes SphinxTranslator.dispatch_departure calling depart_Text for python-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes SphinxTranslator.dispatch_visit calling visit_literal for buffer SphinxTranslator.dispatch_visit calling visit_Text for buffer SphinxTranslator.dispatch_departure calling depart_Text for buffer SphinxTranslator.dispatch_departure calling depart_literal for buffer SphinxTranslator.dispatch_visit calling visit_Text for , SphinxTranslator.dispatch_departure calling depart_Text for , SphinxTranslator.dispatch_visit calling visit_literal for memoryview SphinxTranslator.dispatch_visit calling visit_Text for memoryview SphinxTranslator.dispatch_departure calling depart_Text for memoryview SphinxTranslator.dispatch_departure calling depart_literal for memoryview SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for bytearray SphinxTranslator.dispatch_visit calling visit_Text for bytearray SphinxTranslator.dispatch_departure calling depart_Text for bytearray SphinxTranslator.dispatch_departure calling depart_literal for bytearray SphinxTranslator.dispatch_visit calling visit_Text for : SphinxTranslator.dispatch_departure calling depart_Text for : SphinxTranslator.dispatch_departure calling depart_paragraph for python-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz' SphinxTranslator.dispatch_visit calling visit_paragraph for Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode. SphinxTranslator.dispatch_visit calling visit_Text for Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both SphinxTranslator.dispatch_departure calling depart_Text for Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both SphinxTranslator.dispatch_visit calling visit_literal for str SphinxTranslator.dispatch_visit calling visit_Text for str SphinxTranslator.dispatch_departure calling depart_Text for str SphinxTranslator.dispatch_departure calling depart_literal for str SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for unicode SphinxTranslator.dispatch_visit calling visit_Text for unicode SphinxTranslator.dispatch_departure calling depart_Text for unicode SphinxTranslator.dispatch_departure calling depart_literal for unicode SphinxTranslator.dispatch_visit calling visit_Text for types, whereas in Python 3.x we can only compress ‘binary’ data which does SphinxTranslator.dispatch_departure calling depart_Text for types, whereas in Python 3.x we can only compress ‘binary’ data which does SphinxTranslator.dispatch_visit calling visit_emphasis for not SphinxTranslator.dispatch_visit calling visit_Text for not SphinxTranslator.dispatch_departure calling depart_Text for not SphinxTranslator.dispatch_departure calling depart_emphasis for not SphinxTranslator.dispatch_visit calling visit_Text for include SphinxTranslator.dispatch_departure calling depart_Text for include SphinxTranslator.dispatch_visit calling visit_literal for unicode SphinxTranslator.dispatch_visit calling visit_Text for unicode SphinxTranslator.dispatch_departure calling depart_Text for unicode SphinxTranslator.dispatch_departure calling depart_literal for unicode SphinxTranslator.dispatch_visit calling visit_Text for . SphinxTranslator.dispatch_departure calling depart_Text for . SphinxTranslator.dispatch_departure calling depart_paragraph for Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode. SphinxTranslator.dispatch_departure calling depart_section for
Supporting the buffer interfacepython-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray:>>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz'Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode.
SphinxTranslator.dispatch_visit calling visit_section for
Packaging NumPy arraysWant to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way:>>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/sAlthough this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time.
SphinxTranslator.dispatch_visit calling visit_title for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_Text for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_Text for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_title for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_paragraph for Want to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way: SphinxTranslator.dispatch_visit calling visit_Text for Want to use SphinxTranslator.dispatch_departure calling depart_Text for Want to use SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? SphinxTranslator.dispatch_departure calling depart_Text for to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for comes with the SphinxTranslator.dispatch_departure calling depart_Text for comes with the SphinxTranslator.dispatch_visit calling visit_literal for pack_array SphinxTranslator.dispatch_visit calling visit_Text for pack_array SphinxTranslator.dispatch_departure calling depart_Text for pack_array SphinxTranslator.dispatch_departure calling depart_literal for pack_array SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for unpack_array SphinxTranslator.dispatch_visit calling visit_Text for unpack_array SphinxTranslator.dispatch_departure calling depart_Text for unpack_array SphinxTranslator.dispatch_departure calling depart_literal for unpack_array SphinxTranslator.dispatch_visit calling visit_Text for to perform this in a handy way: SphinxTranslator.dispatch_departure calling depart_Text for to perform this in a handy way: SphinxTranslator.dispatch_departure calling depart_paragraph for Want to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/s SphinxTranslator.dispatch_visit calling visit_paragraph for Although this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time. SphinxTranslator.dispatch_visit calling visit_Text for Although this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time. SphinxTranslator.dispatch_departure calling depart_Text for Although this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time. SphinxTranslator.dispatch_departure calling depart_paragraph for Although this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time. SphinxTranslator.dispatch_departure calling depart_section for
Packaging NumPy arraysWant to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way:>>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/sAlthough this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time.
SphinxTranslator.dispatch_visit calling visit_section for
Compressing from a data pointerFor avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action:>>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() TrueAs you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size).On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization.blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as:>>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True)
SphinxTranslator.dispatch_visit calling visit_title for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_Text for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_Text for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_title for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_paragraph for For avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action: SphinxTranslator.dispatch_visit calling visit_Text for For avoiding the data copy problem in the previous section, SphinxTranslator.dispatch_departure calling depart_Text for For avoiding the data copy problem in the previous section, SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for comes with a couple of lower-level functions: SphinxTranslator.dispatch_departure calling depart_Text for comes with a couple of lower-level functions: SphinxTranslator.dispatch_visit calling visit_literal for compress_ptr SphinxTranslator.dispatch_visit calling visit_Text for compress_ptr SphinxTranslator.dispatch_departure calling depart_Text for compress_ptr SphinxTranslator.dispatch_departure calling depart_literal for compress_ptr SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for decompress_ptr SphinxTranslator.dispatch_visit calling visit_Text for decompress_ptr SphinxTranslator.dispatch_departure calling depart_Text for decompress_ptr SphinxTranslator.dispatch_departure calling depart_literal for decompress_ptr SphinxTranslator.dispatch_visit calling visit_Text for . Here are they in action: SphinxTranslator.dispatch_departure calling depart_Text for . Here are they in action: SphinxTranslator.dispatch_departure calling depart_paragraph for For avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() True SphinxTranslator.dispatch_visit calling visit_paragraph for As you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size). SphinxTranslator.dispatch_visit calling visit_Text for As you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size). SphinxTranslator.dispatch_departure calling depart_Text for As you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size). SphinxTranslator.dispatch_departure calling depart_paragraph for As you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size). SphinxTranslator.dispatch_visit calling visit_paragraph for On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization. SphinxTranslator.dispatch_visit calling visit_Text for On the other hand, and contrarily to the SphinxTranslator.dispatch_departure calling depart_Text for On the other hand, and contrarily to the SphinxTranslator.dispatch_visit calling visit_literal for pack_array SphinxTranslator.dispatch_visit calling visit_Text for pack_array SphinxTranslator.dispatch_departure calling depart_Text for pack_array SphinxTranslator.dispatch_departure calling depart_literal for pack_array SphinxTranslator.dispatch_visit calling visit_Text for / SphinxTranslator.dispatch_departure calling depart_Text for / SphinxTranslator.dispatch_visit calling visit_literal for unpack_array SphinxTranslator.dispatch_visit calling visit_Text for unpack_array SphinxTranslator.dispatch_departure calling depart_Text for unpack_array SphinxTranslator.dispatch_departure calling depart_literal for unpack_array SphinxTranslator.dispatch_visit calling visit_Text for method, the SphinxTranslator.dispatch_departure calling depart_Text for method, the SphinxTranslator.dispatch_visit calling visit_literal for compress_ptr SphinxTranslator.dispatch_visit calling visit_Text for compress_ptr SphinxTranslator.dispatch_departure calling depart_Text for compress_ptr SphinxTranslator.dispatch_departure calling depart_literal for compress_ptr SphinxTranslator.dispatch_visit calling visit_Text for / SphinxTranslator.dispatch_departure calling depart_Text for / SphinxTranslator.dispatch_visit calling visit_literal for decompress_ptr SphinxTranslator.dispatch_visit calling visit_Text for decompress_ptr SphinxTranslator.dispatch_departure calling depart_Text for decompress_ptr SphinxTranslator.dispatch_departure calling depart_literal for decompress_ptr SphinxTranslator.dispatch_visit calling visit_Text for functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization. SphinxTranslator.dispatch_departure calling depart_Text for functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization. SphinxTranslator.dispatch_departure calling depart_paragraph for On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization. SphinxTranslator.dispatch_visit calling visit_paragraph for blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as: SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for has a maximum SphinxTranslator.dispatch_departure calling depart_Text for has a maximum SphinxTranslator.dispatch_visit calling visit_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for blocksize SphinxTranslator.dispatch_departure calling depart_Text for blocksize SphinxTranslator.dispatch_departure calling depart_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for of 2**31 bytes = 2 GB. Larger SphinxTranslator.dispatch_departure calling depart_Text for of 2**31 bytes = 2 GB. Larger SphinxTranslator.dispatch_visit calling visit_literal for numpy SphinxTranslator.dispatch_visit calling visit_Text for numpy SphinxTranslator.dispatch_departure calling depart_Text for numpy SphinxTranslator.dispatch_departure calling depart_literal for numpy SphinxTranslator.dispatch_visit calling visit_Text for arrays must be chunked by slicing, for example as: SphinxTranslator.dispatch_departure calling depart_Text for arrays must be chunked by slicing, for example as: SphinxTranslator.dispatch_departure calling depart_paragraph for blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as: SphinxTranslator.dispatch_visit calling visit_literal_block for >>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) SphinxTranslator.dispatch_departure calling depart_section for
Compressing from a data pointerFor avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action:>>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() TrueAs you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size).On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization.blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as:>>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True)
SphinxTranslator.dispatch_visit calling visit_section for
Fine-tuning compression parametersThere are a number of parameters that affect the de/compression bandwidth achieved by blosc:The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py.
SphinxTranslator.dispatch_visit calling visit_title for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_Text for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_Text for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_title for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_paragraph for There are a number of parameters that affect the de/compression bandwidth achieved by blosc: SphinxTranslator.dispatch_visit calling visit_Text for There are a number of parameters that affect the de/compression bandwidth achieved by SphinxTranslator.dispatch_departure calling depart_Text for There are a number of parameters that affect the de/compression bandwidth achieved by SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for : SphinxTranslator.dispatch_departure calling depart_Text for : SphinxTranslator.dispatch_departure calling depart_paragraph for There are a number of parameters that affect the de/compression bandwidth achieved by blosc: SphinxTranslator.dispatch_visit calling visit_bullet_list for The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py. SphinxTranslator.dispatch_visit calling visit_list_item for The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module. SphinxTranslator.dispatch_visit calling visit_paragraph for The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module. SphinxTranslator.dispatch_visit calling visit_Text for The information content of the underlying data, and SphinxTranslator.dispatch_departure calling depart_Text for The information content of the underlying data, and SphinxTranslator.dispatch_visit calling visit_literal for chunksize SphinxTranslator.dispatch_visit calling visit_Text for chunksize SphinxTranslator.dispatch_departure calling depart_Text for chunksize SphinxTranslator.dispatch_departure calling depart_literal for chunksize SphinxTranslator.dispatch_visit calling visit_Text for : the size of the data in each call to SphinxTranslator.dispatch_departure calling depart_Text for : the size of the data in each call to SphinxTranslator.dispatch_visit calling visit_literal for compress SphinxTranslator.dispatch_visit calling visit_Text for compress SphinxTranslator.dispatch_departure calling depart_Text for compress SphinxTranslator.dispatch_departure calling depart_literal for compress SphinxTranslator.dispatch_visit calling visit_Text for and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the SphinxTranslator.dispatch_departure calling depart_Text for and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the SphinxTranslator.dispatch_visit calling visit_literal for bloscpack SphinxTranslator.dispatch_visit calling visit_Text for bloscpack SphinxTranslator.dispatch_departure calling depart_Text for bloscpack SphinxTranslator.dispatch_departure calling depart_literal for bloscpack SphinxTranslator.dispatch_visit calling visit_Text for module. SphinxTranslator.dispatch_departure calling depart_Text for module. SphinxTranslator.dispatch_departure calling depart_paragraph for The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module. SphinxTranslator.dispatch_departure calling depart_list_item for The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module. SphinxTranslator.dispatch_visit calling visit_list_item for n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores. SphinxTranslator.dispatch_visit calling visit_paragraph for n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores. SphinxTranslator.dispatch_visit calling visit_literal for n_threads SphinxTranslator.dispatch_visit calling visit_Text for n_threads SphinxTranslator.dispatch_departure calling depart_Text for n_threads SphinxTranslator.dispatch_departure calling depart_literal for n_threads SphinxTranslator.dispatch_visit calling visit_Text for : The number of threads to spawn inside SphinxTranslator.dispatch_departure calling depart_Text for : The number of threads to spawn inside SphinxTranslator.dispatch_visit calling visit_literal for c-blosc SphinxTranslator.dispatch_visit calling visit_Text for c-blosc SphinxTranslator.dispatch_departure calling depart_Text for c-blosc SphinxTranslator.dispatch_departure calling depart_literal for c-blosc SphinxTranslator.dispatch_visit calling visit_Text for . SphinxTranslator.dispatch_departure calling depart_Text for . SphinxTranslator.dispatch_visit calling visit_literal for n_threads SphinxTranslator.dispatch_visit calling visit_Text for n_threads SphinxTranslator.dispatch_departure calling depart_Text for n_threads SphinxTranslator.dispatch_departure calling depart_literal for n_threads SphinxTranslator.dispatch_visit calling visit_Text for may be changed by calling SphinxTranslator.dispatch_departure calling depart_Text for may be changed by calling SphinxTranslator.dispatch_visit calling visit_literal for blosc.set_nthreads(16) SphinxTranslator.dispatch_visit calling visit_Text for blosc.set_nthreads(16) SphinxTranslator.dispatch_departure calling depart_Text for blosc.set_nthreads(16) SphinxTranslator.dispatch_departure calling depart_literal for blosc.set_nthreads(16) SphinxTranslator.dispatch_visit calling visit_Text for for example. SphinxTranslator.dispatch_departure calling depart_Text for for example. SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores. SphinxTranslator.dispatch_departure calling depart_Text for performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores. SphinxTranslator.dispatch_departure calling depart_paragraph for n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores. SphinxTranslator.dispatch_departure calling depart_list_item for n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores. SphinxTranslator.dispatch_visit calling visit_list_item for blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization. SphinxTranslator.dispatch_visit calling visit_paragraph for blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization. SphinxTranslator.dispatch_visit calling visit_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for blocksize SphinxTranslator.dispatch_departure calling depart_Text for blocksize SphinxTranslator.dispatch_departure calling depart_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for : is the size of each continuously memory-element that is compressed, in bytes. Normally SphinxTranslator.dispatch_departure calling depart_Text for : is the size of each continuously memory-element that is compressed, in bytes. Normally SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling SphinxTranslator.dispatch_departure calling depart_Text for attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling SphinxTranslator.dispatch_visit calling visit_literal for blosc.set_blocksize( size_in_bytes ) SphinxTranslator.dispatch_visit calling visit_Text for blosc.set_blocksize( size_in_bytes ) SphinxTranslator.dispatch_departure calling depart_Text for blosc.set_blocksize( size_in_bytes ) SphinxTranslator.dispatch_departure calling depart_literal for blosc.set_blocksize( size_in_bytes ) SphinxTranslator.dispatch_visit calling visit_Text for for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization. SphinxTranslator.dispatch_departure calling depart_Text for for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization. SphinxTranslator.dispatch_departure calling depart_paragraph for blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization. SphinxTranslator.dispatch_departure calling depart_list_item for blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization. SphinxTranslator.dispatch_visit calling visit_list_item for shuffle: as discussed above the shuffle mode can substantially improve compression ratios when the SphinxTranslator.dispatch_visit calling visit_paragraph for shuffle: as discussed above the shuffle mode can substantially improve compression ratios when the SphinxTranslator.dispatch_visit calling visit_literal for shuffle SphinxTranslator.dispatch_visit calling visit_Text for shuffle SphinxTranslator.dispatch_departure calling depart_Text for shuffle SphinxTranslator.dispatch_departure calling depart_literal for shuffle SphinxTranslator.dispatch_visit calling visit_Text for : as discussed above the SphinxTranslator.dispatch_departure calling depart_Text for : as discussed above the SphinxTranslator.dispatch_visit calling visit_literal for shuffle SphinxTranslator.dispatch_visit calling visit_Text for shuffle SphinxTranslator.dispatch_departure calling depart_Text for shuffle SphinxTranslator.dispatch_departure calling depart_literal for shuffle SphinxTranslator.dispatch_visit calling visit_Text for mode can substantially improve compression ratios when the SphinxTranslator.dispatch_departure calling depart_Text for mode can substantially improve compression ratios when the SphinxTranslator.dispatch_departure calling depart_paragraph for shuffle: as discussed above the shuffle mode can substantially improve compression ratios when the SphinxTranslator.dispatch_departure calling depart_list_item for shuffle: as discussed above the shuffle mode can substantially improve compression ratios when the SphinxTranslator.dispatch_visit calling visit_list_item for clevel: the compression level called for the algorithm. Called as an argument to compress and similar functions. SphinxTranslator.dispatch_visit calling visit_paragraph for clevel: the compression level called for the algorithm. Called as an argument to compress and similar functions. SphinxTranslator.dispatch_visit calling visit_literal for clevel SphinxTranslator.dispatch_visit calling visit_Text for clevel SphinxTranslator.dispatch_departure calling depart_Text for clevel SphinxTranslator.dispatch_departure calling depart_literal for clevel SphinxTranslator.dispatch_visit calling visit_Text for : the compression level called for the algorithm. Called as an argument to SphinxTranslator.dispatch_departure calling depart_Text for : the compression level called for the algorithm. Called as an argument to SphinxTranslator.dispatch_visit calling visit_literal for compress SphinxTranslator.dispatch_visit calling visit_Text for compress SphinxTranslator.dispatch_departure calling depart_Text for compress SphinxTranslator.dispatch_departure calling depart_literal for compress SphinxTranslator.dispatch_visit calling visit_Text for and similar functions. SphinxTranslator.dispatch_departure calling depart_Text for and similar functions. SphinxTranslator.dispatch_departure calling depart_paragraph for clevel: the compression level called for the algorithm. Called as an argument to compress and similar functions. SphinxTranslator.dispatch_departure calling depart_list_item for clevel: the compression level called for the algorithm. Called as an argument to compress and similar functions. SphinxTranslator.dispatch_visit calling visit_list_item for cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions. SphinxTranslator.dispatch_visit calling visit_paragraph for cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions. SphinxTranslator.dispatch_visit calling visit_literal for cname SphinxTranslator.dispatch_visit calling visit_Text for cname SphinxTranslator.dispatch_departure calling depart_Text for cname SphinxTranslator.dispatch_departure calling depart_literal for cname SphinxTranslator.dispatch_visit calling visit_Text for : the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller SphinxTranslator.dispatch_departure calling depart_Text for : the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller SphinxTranslator.dispatch_visit calling visit_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for blocksize SphinxTranslator.dispatch_departure calling depart_Text for blocksize SphinxTranslator.dispatch_departure calling depart_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for and does not slow significantly with SphinxTranslator.dispatch_departure calling depart_Text for and does not slow significantly with SphinxTranslator.dispatch_visit calling visit_literal for clevel SphinxTranslator.dispatch_visit calling visit_Text for clevel SphinxTranslator.dispatch_departure calling depart_Text for clevel SphinxTranslator.dispatch_departure calling depart_literal for clevel SphinxTranslator.dispatch_visit calling visit_Text for . In comparison SphinxTranslator.dispatch_departure calling depart_Text for . In comparison SphinxTranslator.dispatch_visit calling visit_literal for zlib SphinxTranslator.dispatch_visit calling visit_Text for zlib SphinxTranslator.dispatch_departure calling depart_Text for zlib SphinxTranslator.dispatch_departure calling depart_literal for zlib SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for zstd SphinxTranslator.dispatch_visit calling visit_Text for zstd SphinxTranslator.dispatch_departure calling depart_Text for zstd SphinxTranslator.dispatch_departure calling depart_literal for zstd SphinxTranslator.dispatch_visit calling visit_Text for both slow substantially with increasing SphinxTranslator.dispatch_departure calling depart_Text for both slow substantially with increasing SphinxTranslator.dispatch_visit calling visit_literal for clevel SphinxTranslator.dispatch_visit calling visit_Text for clevel SphinxTranslator.dispatch_departure calling depart_Text for clevel SphinxTranslator.dispatch_departure calling depart_literal for clevel SphinxTranslator.dispatch_visit calling visit_Text for , with an inflection point around SphinxTranslator.dispatch_departure calling depart_Text for , with an inflection point around SphinxTranslator.dispatch_visit calling visit_literal for clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress SphinxTranslator.dispatch_visit calling visit_Text for clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress SphinxTranslator.dispatch_departure calling depart_Text for clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress SphinxTranslator.dispatch_departure calling depart_literal for clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress SphinxTranslator.dispatch_visit calling visit_Text for and similar functions. SphinxTranslator.dispatch_departure calling depart_Text for and similar functions. SphinxTranslator.dispatch_departure calling depart_paragraph for cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions. SphinxTranslator.dispatch_departure calling depart_list_item for cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions. SphinxTranslator.dispatch_visit calling visit_list_item for releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py. SphinxTranslator.dispatch_visit calling visit_paragraph for releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py. SphinxTranslator.dispatch_visit calling visit_literal for releasegil SphinxTranslator.dispatch_visit calling visit_Text for releasegil SphinxTranslator.dispatch_departure calling depart_Text for releasegil SphinxTranslator.dispatch_departure calling depart_literal for releasegil SphinxTranslator.dispatch_visit calling visit_Text for : optionally the Python Global Interpreter Lock (GIL) can be turned off during SphinxTranslator.dispatch_departure calling depart_Text for : optionally the Python Global Interpreter Lock (GIL) can be turned off during SphinxTranslator.dispatch_visit calling visit_literal for c-blosc SphinxTranslator.dispatch_visit calling visit_Text for c-blosc SphinxTranslator.dispatch_departure calling depart_Text for c-blosc SphinxTranslator.dispatch_departure calling depart_literal for c-blosc SphinxTranslator.dispatch_visit calling visit_Text for operations by calling SphinxTranslator.dispatch_departure calling depart_Text for operations by calling SphinxTranslator.dispatch_visit calling visit_literal for blosc.set_releasegil(True) SphinxTranslator.dispatch_visit calling visit_Text for blosc.set_releasegil(True) SphinxTranslator.dispatch_departure calling depart_Text for blosc.set_releasegil(True) SphinxTranslator.dispatch_departure calling depart_literal for blosc.set_releasegil(True) SphinxTranslator.dispatch_visit calling visit_Text for . The default is off as there is a small overhead in releasing the GIL which can be significant for a small SphinxTranslator.dispatch_departure calling depart_Text for . The default is off as there is a small overhead in releasing the GIL which can be significant for a small SphinxTranslator.dispatch_visit calling visit_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for blocksize SphinxTranslator.dispatch_departure calling depart_Text for blocksize SphinxTranslator.dispatch_departure calling depart_literal for blocksize SphinxTranslator.dispatch_visit calling visit_Text for . GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python SphinxTranslator.dispatch_departure calling depart_Text for . GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python SphinxTranslator.dispatch_visit calling visit_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for or similar object can be used for parallel processing either with or without SphinxTranslator.dispatch_departure calling depart_Text for or similar object can be used for parallel processing either with or without SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for threads. Initial testing suggests that an equal mix of SphinxTranslator.dispatch_departure calling depart_Text for threads. Initial testing suggests that an equal mix of SphinxTranslator.dispatch_visit calling visit_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for threads is near optimal. I.e. a computer with 16 cores would have 4 SphinxTranslator.dispatch_departure calling depart_Text for threads is near optimal. I.e. a computer with 16 cores would have 4 SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for threads and 4 SphinxTranslator.dispatch_departure calling depart_Text for threads and 4 SphinxTranslator.dispatch_visit calling visit_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for threads. An example of combining SphinxTranslator.dispatch_departure calling depart_Text for threads. An example of combining SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for and SphinxTranslator.dispatch_departure calling depart_Text for and SphinxTranslator.dispatch_visit calling visit_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_Text for ThreadPool SphinxTranslator.dispatch_departure calling depart_literal for ThreadPool SphinxTranslator.dispatch_visit calling visit_Text for may be found in SphinxTranslator.dispatch_departure calling depart_Text for may be found in SphinxTranslator.dispatch_visit calling visit_literal for bench/threadpool.py SphinxTranslator.dispatch_visit calling visit_Text for bench/threadpool.py SphinxTranslator.dispatch_departure calling depart_Text for bench/threadpool.py SphinxTranslator.dispatch_departure calling depart_literal for bench/threadpool.py SphinxTranslator.dispatch_visit calling visit_Text for . SphinxTranslator.dispatch_departure calling depart_Text for . SphinxTranslator.dispatch_departure calling depart_paragraph for releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py. SphinxTranslator.dispatch_departure calling depart_list_item for releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py. SphinxTranslator.dispatch_departure calling depart_bullet_list for The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py. SphinxTranslator.dispatch_departure calling depart_section for
Fine-tuning compression parametersThere are a number of parameters that affect the de/compression bandwidth achieved by blosc:The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py.
SphinxTranslator.dispatch_visit calling visit_section for
Links to external discussions on <literal>blosc</literal> optimizationSynthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles
SphinxTranslator.dispatch_visit calling visit_title for Links to external discussions on <literal>blosc</literal> optimization SphinxTranslator.dispatch_visit calling visit_Text for Links to external discussions on SphinxTranslator.dispatch_departure calling depart_Text for Links to external discussions on SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for optimization SphinxTranslator.dispatch_departure calling depart_Text for optimization SphinxTranslator.dispatch_departure calling depart_title for Links to external discussions on <literal>blosc</literal> optimization SphinxTranslator.dispatch_visit calling visit_bullet_list for Synthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles SphinxTranslator.dispatch_visit calling visit_list_item for Synthetic Benchmarks by Francesc Alted SphinxTranslator.dispatch_visit calling visit_paragraph for Synthetic Benchmarks by Francesc Alted SphinxTranslator.dispatch_visit calling visit_reference for Synthetic Benchmarks SphinxTranslator.dispatch_visit calling visit_Text for Synthetic Benchmarks SphinxTranslator.dispatch_departure calling depart_Text for Synthetic Benchmarks SphinxTranslator.dispatch_departure calling depart_reference for Synthetic Benchmarks SphinxTranslator.dispatch_visit calling visit_target for SphinxTranslator.dispatch_departure calling depart_target for SphinxTranslator.dispatch_visit calling visit_Text for by Francesc Alted SphinxTranslator.dispatch_departure calling depart_Text for by Francesc Alted SphinxTranslator.dispatch_departure calling depart_paragraph for Synthetic Benchmarks by Francesc Alted SphinxTranslator.dispatch_departure calling depart_list_item for Synthetic Benchmarks by Francesc Alted SphinxTranslator.dispatch_visit calling visit_list_item for Genotype compressor benchmark by Alistair Miles SphinxTranslator.dispatch_visit calling visit_paragraph for Genotype compressor benchmark by Alistair Miles SphinxTranslator.dispatch_visit calling visit_reference for Genotype compressor benchmark SphinxTranslator.dispatch_visit calling visit_Text for Genotype compressor benchmark SphinxTranslator.dispatch_departure calling depart_Text for Genotype compressor benchmark SphinxTranslator.dispatch_departure calling depart_reference for Genotype compressor benchmark SphinxTranslator.dispatch_visit calling visit_target for SphinxTranslator.dispatch_departure calling depart_target for SphinxTranslator.dispatch_visit calling visit_Text for by Alistair Miles SphinxTranslator.dispatch_departure calling depart_Text for by Alistair Miles SphinxTranslator.dispatch_departure calling depart_paragraph for Genotype compressor benchmark by Alistair Miles SphinxTranslator.dispatch_departure calling depart_list_item for Genotype compressor benchmark by Alistair Miles SphinxTranslator.dispatch_departure calling depart_bullet_list for Synthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles SphinxTranslator.dispatch_departure calling depart_section for
Links to external discussions on <literal>blosc</literal> optimizationSynthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles
SphinxTranslator.dispatch_departure calling depart_section for
TutorialsUsing python-blosc (or just blosc, because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the zlib module included in the standard Python library.Here are some examples on how to use it. For the full documentation, please refer to the Library Reference section.Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz.
Compressing and decompressing with <literal>blosc</literal>Let’s start creating a NumPy array with 80 MB full of data:>>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytesand let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities):>>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlibbut Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default):>>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlibIt is important to note that this is quite more than the speed of a memcpy() in this machine:>>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/swhich means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises.Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib:>>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlibThe reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs.Now, let’s have a look at the compression ratios:>>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratioHere we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string:>>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib:>>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLibHere it is a plot with the different compression ratios achieved:The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data.How about decompression?:>>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlibHere we see a couple of things:All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression).The next plot summarizes the speed benchmarks above:These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation).Finally, here it is the way to discover all the internal codecs inside your Blosc package:>>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library.
Using different filtersIn the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets.Here it is an example using the SHUFFLE filter:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533Here there is another example using BITSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffleYou can also deactivate filters completely with NOSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original bufferSo you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one.
Supporting the buffer interfacepython-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray:>>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz'Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode.
Packaging NumPy arraysWant to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way:>>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/sAlthough this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time.
Compressing from a data pointerFor avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action:>>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() TrueAs you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size).On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization.blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as:>>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True)
Fine-tuning compression parametersThere are a number of parameters that affect the de/compression bandwidth achieved by blosc:The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py.
Links to external discussions on <literal>blosc</literal> optimizationSynthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles
SphinxTranslator.dispatch_departure calling depart_document for
TutorialsUsing python-blosc (or just blosc, because we are going to talk always on how to use it in a Python environment) is pretty easy. It basically mimics the API of the zlib module included in the standard Python library.Here are some examples on how to use it. For the full documentation, please refer to the Library Reference section.Most of the times in this tutorial have been obtained using a VM with 2 cores on top of a Intel(R) Core(TM) i5-3380M CPU @ 2.90GHz.
Compressing and decompressing with <literal>blosc</literal>Let’s start creating a NumPy array with 80 MB full of data:>>> import numpy as np >>> a = np.linspace(0, 100, 1e7) >>> bytes_array = a.tostring() # get an array of bytesand let’s compare Blosc operation with zlib (please note that we are using IPython for leveraging its timing capabilities):>>> import zlib >>> %timeit zlib.compress(bytes_array) 1 loops, best of 3: 4.65 s per loop # ~ 17 MB/s >>> import blosc >>> %timeit blosc.compress(bytes_array, typesize=8) 100 loops, best of 3: 17.7 ms per loop # ~ 4.5 GB/s and 260x faster than zlibbut Blosc can use different codecs under the hood. Let’s try the LZ4 codec (instead of the BloscLZ which is the default):>>> %timeit blosc.compress(bytes_array, typesize=8, cname='lz4') 100 loops, best of 3: 8.77 ms per loop # ~ 9.1 GB/s and 530x faster than zlibIt is important to note that this is quite more than the speed of a memcpy() in this machine:>>> %timeit a.copy() 10 loops, best of 3: 27.2 ms per loop # ~ 2.9 GB/swhich means that both BloscLZ and LZ4 codecs can be faster than memcpy(), just as the Blosc slogan promises.Blosc also comes with the ZLib codec too, and it actually runs faster than the naked zlib:>>> %timeit blosc.compress(bytes_array, typesize=8, cname='zlib') 10 loops, best of 3: 139 ms per loop # ~ 580 MB/s and 33x faster than zlibThe reason why the internal Zlib codec in Blosc is faster than the ‘naked’ one is 1) that Blosc can use multithreading (using 8 threads here) and 2) that Blosc splits the data to compress in smaller blocks that are more friendly to the caches in modern CPUs.Now, let’s have a look at the compression ratios:>>> zpacked = zlib.compress(bytes_array); len(zpacked) 52994692 >>> round(len(bytes_array) / float(len(zpacked)), 3) 1.51 # zlib achieves a 1.5x compression ratio >>> blzpacked = blosc.compress(bytes_array, typesize=8, cname='blosclz'); len(blzpacked) 6986533 >>> round(len(bytes_array) / float(len(blzpacked)), 3) 11.451 # BloscLZ codec reaches more than 11x compression ratio >>> lz4packed = blosc.compress(bytes_array, typesize=8, cname='lz4'); len(lz4packed) 3716015 >>> round(len(bytes_array) / float(len(lz4packed)), 3) 21.528 # LZ4 codec reaches more than 21x compression ratioHere we can see how Blosc, with its different codecs, is very efficient compressing this kind of binary data. It is important to note that the codecs alone are not the only responsible for high compression ratios. See what a naked LZ4 codec can do on the same string:>>> import lz4 >>> lz4_packed = lz4.compress(bytes_array); len(lz4_packed) 80309133 >>> round(len(bytes_array) / float(len(lz4_packed)), 3) 0.996That is, the naked LZ4 codec cannot compress the array of bytes at all, whereas through Blosc LZ4 can compress quite a lot. In fact, this difference in compression also happens with ZLib:>>> zlibpacked = blosc.compress(bytes_array, typesize=8, cname='zlib'); len(zlibpacked) 875274 >>> round(len(zpacked) / float(len(zlibpacked)), 3) 60.546 # ZLib codec reaches 60x more compression than naked ZLibHere it is a plot with the different compression ratios achieved:The explanation for this apparently shocking result is that Blosc uses filters (SHUFFLE and BITSHUFFLE currently, but the list can increase more in the future) prior to the compression stage and these allow in general for better compression ratios when using binary data.How about decompression?:>>> %timeit zlib.decompress(zpacked) 1 loops, best of 3: 275 ms per loop # ~ 290 MB/s >>> %timeit blosc.decompress(blzpacked) 100 loops, best of 3: 17.3 ms per loop # ~ 4.3 GB/s and 16x faster than zlib >>> %timeit blosc.decompress(lz4packed) 10 loops, best of 3: 41.9 ms per loop # ~ 1.9 GB/s and 6.5x faster than zlib >>> %timeit blosc.decompress(zlibpacked) 10 loops, best of 3: 40.7 ms per loop # ~ 2.0 GB/S and 6.8x faster than zlibHere we see a couple of things:All the internal codecs in Blosc are way faster than naked ZLibThe fastest codec for decompressing here is BloscLZ (remember that LZ4 was the fastest for compression).The next plot summarizes the speed benchmarks above:These results should reinforce the idea that there is not a single codec that wins in all areas (compression ratio, compression speed and decompression speed) and each has its pro’s and con’s. It is up to the user to choose whatever fits better to him (hint: there is no replacement for experimentation).Finally, here it is the way to discover all the internal codecs inside your Blosc package:>>> blosc.cnames ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']Note: the actual list of codecs may change depening on how you have compiled the underlying C-Blosc library.
Using different filtersIn the same way that you can use different codecs inside Blosc, you can use different filters too (currently SHUFFLE and BITSHUFLE). These allow the integrated compressors to compress more efficiently or not, depending on your datasets.Here it is an example using the SHUFFLE filter:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.SHUFFLE) CPU times: user 112 ms, sys: 4 ms, total: 116 ms Wall time: 29.9 ms >>> len(bpacked) 6986533Here there is another example using BITSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.BITSHUFFLE) CPU times: user 120 ms, sys: 8 ms, total: 128 ms Wall time: 32.9 ms >>> len(bpacked) 5942257 # ~ 1.2x smaller than blosclz/shuffleYou can also deactivate filters completely with NOSHUFFLE:>>> %time bpacked = blosc.compress(bytes_array, shuffle=blosc.NOSHUFFLE) CPU times: user 416 ms, sys: 8 ms, total: 424 ms Wall time: 107 ms >>> len(bpacked) 74323591 # just a 7% of compression wrt the original bufferSo you have quite a bit of flexibility on choosing different codecs and filters inside Blosc. Again, depending on the dataset you have and the requeriments on performance, you may want to experiment a bit before sticking with your preferred one.
Supporting the buffer interfacepython-blosc supports compressing and decompressing from any bytes-like object that supports the buffer-interface: this includes buffer, memoryview and bytearray:>>> input_bytes = b"abcdefghijklmnopqrstuvwxyz" >>> blosc.compress(input_bytes, typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(memoryview(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> blosc.compress(bytearray(input_bytes), typesize=1) '\x02\x01\x03\x01\x1a\x00\x00\x00\x1a\x00\x00\x00*\x00\x00\x00abcdefghijklmnopqrstuvwxyz' >>> compressed = blosc.compress(input_bytes, typesize=1) >>> blosc.decompress(compressed) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(memoryview(compressed)) 'abcdefghijklmnopqrstuvwxyz' >>> blosc.decompress(bytearray(compressed)) 'abcdefghijklmnopqrstuvwxyz'Note however, that there are subtle differences between Python 2.x and 3.x. For example, in Python 2.x we can compress/decompress both str and unicode types, whereas in Python 3.x we can only compress ‘binary’ data which does not include unicode.
Packaging NumPy arraysWant to use blosc to compress and decompress NumPy objects without having to worry about passing the typesize for optimal compression, or having to create the final container for decompression? blosc comes with the pack_array and unpack_array to perform this in a handy way:>>> a = np.linspace(0, 100, 1e7) >>> packed = blosc.pack_array(a) >>> %timeit blosc.pack_array(a) 10 loops, best of 3: 104 ms per loop # ~ 770 MB/s >>> %timeit blosc.unpack_array(packed) 10 loops, best of 3: 76.3 ms per loop # ~ 1 GB/sAlthough this is a convenient way for compressing/decompressing NumPy arrays, this method uses pickle/unpickle behind the scenes. This step implies additional copies, which takes both memory and time.
Compressing from a data pointerFor avoiding the data copy problem in the previous section, blosc comes with a couple of lower-level functions: compress_ptr and decompress_ptr. Here are they in action:>>> c = blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) >>> %timeit blosc.compress_ptr(a.__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True) 10 loops, best of 3: 17.8 ms per loop # ~ 4.5 GB/s >>> a2 = np.empty(a.size, dtype=a.dtype) >>> %timeit blosc.decompress_ptr(c, a2.__array_interface__['data'][0]) 100 loops, best of 3: 11 ms per loop # ~ 7.3 GB/s >>> (a == a2).all() TrueAs you see, these are really low level functions because you should pass actual pointers where the data is, as well as the size and itemsize (for compression). Needless to say, it is very easy to cause a segfault by passing incorrect parameters to the functions (wrong pointer or wrong size).On the other hand, and contrarily to the pack_array / unpack_array method, the compress_ptr / decompress_ptr functions do not need to make internal copies of the data buffers, so they are extremely fast (as much as the C-Blosc library can be), but you have to provide a container when doing the de-serialization.blosc has a maximum blocksize of 2**31 bytes = 2 GB. Larger numpy arrays must be chunked by slicing, for example as:>>> c = b'' >>> for index in np.arange( a.shape[0] ): c += blosc.compress_ptr(a[index,...].__array_interface__['data'][0], a.size, a.dtype.itemsize, 9, True)
Fine-tuning compression parametersThere are a number of parameters that affect the de/compression bandwidth achieved by blosc:The information content of the underlying data, and chunksize: the size of the data in each call to compress and analogous functions. ‘blosc’ does not manage chunksize itself, but an example implementation can be seen in the bloscpack module.n_threads: The number of threads to spawn inside c-blosc. n_threads may be changed by calling blosc.set_nthreads(16) for example. blosc performance generally scales sub-linearly with the number of threads with a coefficient roughly around 0.5-0.67. I.e. the expected performance compared to a single thread is 1.0 / (0.6*n_threads). For systems with hyper-threading the optimum number of threads is usually a small over-subscription of the number of _physical_ (not virtual) cores.blocksize: is the size of each continuously memory-element that is compressed, in bytes. Normally blosc attempts to automatically guess the size of each compressed block of data, but the user can set it manually by calling blosc.set_blocksize( size_in_bytes ) for hand optimized situations. Often the L2 cache size (e.g. 256kB for an Intel Haswell) is a good starting point for optimization.shuffle: as discussed above the shuffle mode can substantially improve compression ratios when theclevel: the compression level called for the algorithm. Called as an argument to compress and similar functions.cname: the compressor codec itself. Each codec has its own characteristics that also vary depending on the underlying data. For example, ‘lz4’ tends to prefer smaller blocksize and does not slow significantly with clevel. In comparison zlib and zstd both slow substantially with increasing clevel, with an inflection point around clevel`=4-5 for 'zlib' and `clevel`=2-4 for 'zstd'. Called as an argument to `compress and similar functions.releasegil: optionally the Python Global Interpreter Lock (GIL) can be turned off during c-blosc operations by calling blosc.set_releasegil(True). The default is off as there is a small overhead in releasing the GIL which can be significant for a small blocksize. GIL release is intended to be used in situations where other bounds (such as file or network I/O) are the rate-limiting ones and a Python ThreadPool or similar object can be used for parallel processing either with or without blosc threads. Initial testing suggests that an equal mix of ThreadPool and blosc threads is near optimal. I.e. a computer with 16 cores would have 4 blosc threads and 4 ThreadPool threads. An example of combining blosc and ThreadPool may be found in bench/threadpool.py.
Links to external discussions on <literal>blosc</literal> optimizationSynthetic Benchmarks by Francesc AltedGenotype compressor benchmark by Alistair Miles
SphinxTranslator.dispatch_visit calling visit_document for Library Reference SphinxTranslator.dispatch_visit calling visit_title for Library Reference SphinxTranslator.dispatch_visit calling visit_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_Text for Library Reference SphinxTranslator.dispatch_departure calling depart_title for Library Reference SphinxTranslator.dispatch_departure calling depart_document for Library Reference SphinxTranslator.dispatch_visit calling visit_document for Installation SphinxTranslator.dispatch_visit calling visit_title for Installation SphinxTranslator.dispatch_visit calling visit_Text for Installation SphinxTranslator.dispatch_departure calling depart_Text for Installation SphinxTranslator.dispatch_departure calling depart_title for Installation SphinxTranslator.dispatch_departure calling depart_document for Installation SphinxTranslator.dispatch_visit calling visit_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_Text for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_title for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_departure calling depart_document for Welcome to python-blosc’s documentation! SphinxTranslator.dispatch_visit calling visit_document for Tutorials SphinxTranslator.dispatch_visit calling visit_title for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_title for Tutorials SphinxTranslator.dispatch_departure calling depart_document for Tutorials SphinxTranslator.dispatch_visit calling visit_document for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_bullet_list for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_list_item for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Tutorials SphinxTranslator.dispatch_visit calling visit_reference for Tutorials SphinxTranslator.dispatch_visit calling visit_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_Text for Tutorials SphinxTranslator.dispatch_departure calling depart_reference for Tutorials SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Tutorials SphinxTranslator.dispatch_visit calling visit_bullet_list for Compressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_list_item for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_reference for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_Text for Compressing and decompressing with SphinxTranslator.dispatch_departure calling depart_Text for Compressing and decompressing with SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_departure calling depart_reference for Compressing and decompressing with blosc SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Compressing and decompressing with blosc SphinxTranslator.dispatch_departure calling depart_list_item for Compressing and decompressing with blosc SphinxTranslator.dispatch_visit calling visit_list_item for Using different filters SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Using different filters SphinxTranslator.dispatch_visit calling visit_reference for Using different filters SphinxTranslator.dispatch_visit calling visit_Text for Using different filters SphinxTranslator.dispatch_departure calling depart_Text for Using different filters SphinxTranslator.dispatch_departure calling depart_reference for Using different filters SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Using different filters SphinxTranslator.dispatch_departure calling depart_list_item for Using different filters SphinxTranslator.dispatch_visit calling visit_list_item for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_reference for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_Text for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_Text for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_reference for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Supporting the buffer interface SphinxTranslator.dispatch_departure calling depart_list_item for Supporting the buffer interface SphinxTranslator.dispatch_visit calling visit_list_item for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_reference for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_Text for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_Text for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_reference for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Packaging NumPy arrays SphinxTranslator.dispatch_departure calling depart_list_item for Packaging NumPy arrays SphinxTranslator.dispatch_visit calling visit_list_item for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_reference for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_Text for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_Text for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_reference for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Compressing from a data pointer SphinxTranslator.dispatch_departure calling depart_list_item for Compressing from a data pointer SphinxTranslator.dispatch_visit calling visit_list_item for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_reference for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_Text for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_Text for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_reference for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Fine-tuning compression parameters SphinxTranslator.dispatch_departure calling depart_list_item for Fine-tuning compression parameters SphinxTranslator.dispatch_visit calling visit_list_item for Links to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_compact_paragraph for Links to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_reference for Links to external discussions on blosc optimization SphinxTranslator.dispatch_visit calling visit_Text for Links to external discussions on SphinxTranslator.dispatch_departure calling depart_Text for Links to external discussions on SphinxTranslator.dispatch_visit calling visit_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for blosc SphinxTranslator.dispatch_departure calling depart_Text for blosc SphinxTranslator.dispatch_departure calling depart_literal for blosc SphinxTranslator.dispatch_visit calling visit_Text for optimization SphinxTranslator.dispatch_departure calling depart_Text for optimization SphinxTranslator.dispatch_departure calling depart_reference for Links to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_compact_paragraph for Links to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_list_item for Links to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_bullet_list for Compressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_list_item for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_bullet_list for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization SphinxTranslator.dispatch_departure calling depart_document for TutorialsCompressing and decompressing with bloscUsing different filtersSupporting the buffer interfacePackaging NumPy arraysCompressing from a data pointerFine-tuning compression parametersLinks to external discussions on blosc optimization [app] emitting event: 'html-page-context'('tutorial', 'page.html', {'embedded': False, 'project': 'python-blosc', 'release': '1.5.0', 'versio generating indices... genindex[app] emitting event: 'html-page-context'('genindex', 'genindex.html', {'embedded': False, 'project': 'python-blosc', 'release': '1.5.0', 've py-modindex[app] emitting event: 'html-page-context'('py-modindex', 'domainindex.html', {'embedded': False, 'project': 'python-blosc', 'release': '1.5.0 done [app] emitting event: 'html-collect-pages'() writing additional pages... search[app] emitting event: 'html-page-context'('search', 'search.html', {'embedded': False, 'project': 'python-blosc', 'release': '1.5.0', 'versio done copying images... [ 50%] cratio-blosc-codecs.png copying images... [100%] speed-blosc-codecs.png copying static files... ... done copying extra files... done dumping search index in English (code: en)... done dumping object inventory... done build succeeded, 2 warnings. The HTML pages are in _build/html. [app] emitting event: 'build-finished'(None,) Build finished. The HTML pages are in _build/html. make: Leaving directory '/usr/src/RPM/BUILD/python3-module-blosc-1.5.1/doc' + cp -fR doc/_build/pickle /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/ + /usr/lib/rpm/brp-alt Cleaning files in /usr/src/tmp/python3-module-blosc-buildroot (auto) mode of './usr/lib/python3/site-packages/blosc/blosc_extension.cpython-38.so' changed from 0755 (rwxr-xr-x) to 0644 (rw-r--r--) Verifying and fixing files in /usr/src/tmp/python3-module-blosc-buildroot (binconfig,pkgconfig,libtool,desktop) Checking contents of files in /usr/src/tmp/python3-module-blosc-buildroot/ (default) Compressing files in /usr/src/tmp/python3-module-blosc-buildroot (auto) Adjusting library links in /usr/src/tmp/python3-module-blosc-buildroot ./usr/lib: Verifying ELF objects in /usr/src/tmp/python3-module-blosc-buildroot (arch=normal,fhs=normal,lfs=relaxed,lint=relaxed,rpath=normal,stack=normal,textrel=normal,unresolved=normal) Bytecompiling python modules in /usr/src/tmp/python3-module-blosc-buildroot using /usr/bin/python2.7 Bytecompiling python modules with optimization in /usr/src/tmp/python3-module-blosc-buildroot using /usr/bin/python2.7 -O Bytecompiling python3 modules in /usr/src/tmp/python3-module-blosc-buildroot using /usr/bin/python3 unlink /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/__pycache__/__init__.cpython-38.pyc unlink /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/__pycache__/test.cpython-38.pyc unlink /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/__pycache__/toplevel.cpython-38.pyc unlink /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/__pycache__/version.cpython-38.pyc compile /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/__init__.py compile /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py compile /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/toplevel.py compile /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/version.py Bytecompiling python3 modules with optimization in /usr/src/tmp/python3-module-blosc-buildroot using /usr/bin/python3 -O compile /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/__init__.py compile /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py compile /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/toplevel.py compile /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/version.py Bytecompiling python3 modules with optimization-2 in /usr/src/tmp/python3-module-blosc-buildroot using /usr/bin/python3 -OO compile /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/__init__.py compile /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py compile /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/toplevel.py compile /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/version.py Hardlinking identical .pyc and .opt-?.pyc files './usr/lib/python3/site-packages/blosc/__pycache__/version.cpython-38.opt-1.pyc' => './usr/lib/python3/site-packages/blosc/__pycache__/version.cpython-38.pyc' './usr/lib/python3/site-packages/blosc/__pycache__/version.cpython-38.opt-2.pyc' => './usr/lib/python3/site-packages/blosc/__pycache__/version.cpython-38.opt-1.pyc' './usr/lib/python3/site-packages/blosc/__pycache__/toplevel.cpython-38.opt-1.pyc' => './usr/lib/python3/site-packages/blosc/__pycache__/toplevel.cpython-38.pyc' './usr/lib/python3/site-packages/blosc/__pycache__/test.cpython-38.opt-2.pyc' => './usr/lib/python3/site-packages/blosc/__pycache__/test.cpython-38.opt-1.pyc' './usr/lib/python3/site-packages/blosc/__pycache__/__init__.cpython-38.opt-1.pyc' => './usr/lib/python3/site-packages/blosc/__pycache__/__init__.cpython-38.pyc' './usr/lib/python3/site-packages/blosc/__pycache__/__init__.cpython-38.opt-2.pyc' => './usr/lib/python3/site-packages/blosc/__pycache__/__init__.cpython-38.opt-1.pyc' Hardlinking identical .pyc and .pyo files Processing files: python3-module-blosc-1.5.1-alt3 Executing(%doc): /bin/sh -e /usr/src/tmp/rpm-tmp.84632 + umask 022 + /bin/mkdir -p /usr/src/RPM/BUILD + cd /usr/src/RPM/BUILD + cd python3-module-blosc-1.5.1 + DOCDIR=/usr/src/tmp/python3-module-blosc-buildroot/usr/share/doc/python3-module-blosc-1.5.1 + export DOCDIR + rm -rf /usr/src/tmp/python3-module-blosc-buildroot/usr/share/doc/python3-module-blosc-1.5.1 + /bin/mkdir -p /usr/src/tmp/python3-module-blosc-buildroot/usr/share/doc/python3-module-blosc-1.5.1 + cp -prL ANNOUNCE.rst README.rst RELEASE_NOTES.rst RELEASING.rst bench /usr/src/tmp/python3-module-blosc-buildroot/usr/share/doc/python3-module-blosc-1.5.1 + chmod -R go-w /usr/src/tmp/python3-module-blosc-buildroot/usr/share/doc/python3-module-blosc-1.5.1 + chmod -R a+rX /usr/src/tmp/python3-module-blosc-buildroot/usr/share/doc/python3-module-blosc-1.5.1 + cp -prL doc/_build/html /usr/src/tmp/python3-module-blosc-buildroot/usr/share/doc/python3-module-blosc-1.5.1 + chmod -R go-w /usr/src/tmp/python3-module-blosc-buildroot/usr/share/doc/python3-module-blosc-1.5.1 + chmod -R a+rX /usr/src/tmp/python3-module-blosc-buildroot/usr/share/doc/python3-module-blosc-1.5.1 + exit 0 Finding Provides (using /usr/lib/rpm/find-provides) Executing: /bin/sh -e /usr/src/tmp/rpm-tmp.krwwQC find-provides: running scripts (alternatives,debuginfo,lib,pam,perl,pkgconfig,python,python3,shell) Finding Requires (using /usr/lib/rpm/find-requires) Executing: /bin/sh -e /usr/src/tmp/rpm-tmp.duytfE find-requires: running scripts (cpp,debuginfo,files,lib,pam,perl,pkgconfig,pkgconfiglib,python,python3,rpmlib,shebang,shell,static,symlinks,systemd-services) python3.req: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/__init__.py: skipping atexit python3.req: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/toplevel.py: skipping sys /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/toplevel.py: line=13 IGNORE (for REQ=slight and deep=8) module=cPickle /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/toplevel.py: line=15 IGNORE (for REQ=slight and deep=8) module=pickle /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/toplevel.py: line=699 IGNORE (for REQ=slight and deep=8) module=doctest /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/toplevel.py: line=706 IGNORE (for REQ=slight and deep=8) module=platform /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/toplevel.py: line=730 IGNORE (for REQ=slight and deep=8) module=doctest Provides: python3(blosc), python3(blosc.blosc_extension), python3(blosc.toplevel), python3(blosc.version) Requires: /usr/lib/python3/site-packages, libblosc.so.1 >= set:idARsJ17kGyK2d0ISkrKpT0wXT9TlZ3Q1znRrb72, libc.so.6(GLIBC_2.1.3), libc.so.6(GLIBC_2.4), python3(blosc.blosc_extension) < 0, python3(blosc.test) < 0, python3(distutils.version) < 0, python3(os) < 0, python3.8-ABI, rtld(GNU_HASH) Requires(rpmlib): rpmlib(SetVersions) Finding debuginfo files (using /usr/lib/rpm/find-debuginfo-files) Executing: /bin/sh -e /usr/src/tmp/rpm-tmp.DpSVnG Creating python3-module-blosc-debuginfo package Processing files: python3-module-blosc-tests-1.5.1-alt3 Finding Provides (using /usr/lib/rpm/find-provides) Executing: /bin/sh -e /usr/src/tmp/rpm-tmp.CkUy6E find-provides: running scripts (alternatives,debuginfo,lib,pam,perl,pkgconfig,python,python3,shell) Finding Requires (using /usr/lib/rpm/find-requires) Executing: /bin/sh -e /usr/src/tmp/rpm-tmp.dyW4qD find-requires: running scripts (cpp,debuginfo,files,lib,pam,perl,pkgconfig,pkgconfiglib,python,python3,rpmlib,shebang,shell,static,symlinks,systemd-services) python3.req: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py: skipping sys python3.req: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py: skipping gc /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py: line=16 IGNORE (for REQ=slight and deep=8) module=unittest2 /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py: line=18 IGNORE (for REQ=slight and deep=8) module=unittest /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py: line=21 IGNORE (for REQ=slight and deep=8) module=numpy /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py: line=28 IGNORE (for REQ=slight and deep=8) module=psutil /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py: line=71 IGNORE (for REQ=slight and deep=11) module=numpy /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py: line=99 IGNORE (for REQ=slight and deep=11) module=numpy /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py: line=117 IGNORE (for REQ=slight and deep=11) module=numpy /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py: line=137 IGNORE (for REQ=slight and deep=11) module=numpy /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py: line=299 IGNORE (for REQ=slight and deep=8) module=blosc /usr/lib/rpm/python3.req.py: /usr/src/tmp/python3-module-blosc-buildroot/usr/lib/python3/site-packages/blosc/test.py: line=300 IGNORE (for REQ=slight and deep=8) module=blosc.toplevel Provides: python3(blosc.test) Requires: python3-module-blosc = 1.5.1-alt3, /usr/lib/python3/site-packages, python3(__future__) < 0, python3(blosc) < 0, python3(ctypes) < 0, python3(distutils.version) < 0, python3(os) < 0 Finding debuginfo files (using /usr/lib/rpm/find-debuginfo-files) Executing: /bin/sh -e /usr/src/tmp/rpm-tmp.WKv5OF Processing files: python3-module-blosc-pickles-1.5.1-alt3 Finding Provides (using /usr/lib/rpm/find-provides) Executing: /bin/sh -e /usr/src/tmp/rpm-tmp.HaRQgD find-provides: running scripts (alternatives,debuginfo,lib,pam,perl,pkgconfig,python,python3,shell) Finding Requires (using /usr/lib/rpm/find-requires) Executing: /bin/sh -e /usr/src/tmp/rpm-tmp.Oef8AD find-requires: running scripts (cpp,debuginfo,files,lib,pam,perl,pkgconfig,pkgconfiglib,python,python3,rpmlib,shebang,shell,static,symlinks,systemd-services) Requires: /usr/lib/python3/site-packages Finding debuginfo files (using /usr/lib/rpm/find-debuginfo-files) Executing: /bin/sh -e /usr/src/tmp/rpm-tmp.NdvzyD Processing files: python3-module-blosc-debuginfo-1.5.1-alt3 Finding Provides (using /usr/lib/rpm/find-provides) Executing: /bin/sh -e /usr/src/tmp/rpm-tmp.RdBjcF find-provides: running scripts (debuginfo) Finding Requires (using /usr/lib/rpm/find-requires) Executing: /bin/sh -e /usr/src/tmp/rpm-tmp.wqm40B find-requires: running scripts (debuginfo) Requires: python3-module-blosc = 1.5.1-alt3, debug(libblosc.so.1), debug(libc.so.6) Adding to python3-module-blosc-tests a strict dependency on python3-module-blosc Adding to python3-module-blosc-debuginfo a strict dependency on python3-module-blosc Adding to python3-module-blosc a strict dependency on python3-module-blosc-tests Removing 1 extra deps from python3-module-blosc due to dependency on python3-module-blosc-tests Removing 1 extra deps from python3-module-blosc-tests due to dependency on python3-module-blosc Removing 3 extra deps from python3-module-blosc due to repentancy on python3-module-blosc-tests Wrote: /usr/src/RPM/RPMS/i586/python3-module-blosc-1.5.1-alt3.i586.rpm Wrote: /usr/src/RPM/RPMS/i586/python3-module-blosc-tests-1.5.1-alt3.i586.rpm Wrote: /usr/src/RPM/RPMS/i586/python3-module-blosc-pickles-1.5.1-alt3.i586.rpm Wrote: /usr/src/RPM/RPMS/i586/python3-module-blosc-debuginfo-1.5.1-alt3.i586.rpm 29.78user 2.97system 0:53.58elapsed 61%CPU (0avgtext+0avgdata 39384maxresident)k 0inputs+0outputs (0major+379675minor)pagefaults 0swaps /.out/python3-module-blosc-1.5.1-alt3.i586.rpm: bad symbols in the license tag: / /.out/python3-module-blosc-tests-1.5.1-alt3.i586.rpm: bad symbols in the license tag: / /.out/python3-module-blosc-pickles-1.5.1-alt3.i586.rpm: bad symbols in the license tag: / /.out/python3-module-blosc-debuginfo-1.5.1-alt3.i586.rpm: bad symbols in the license tag: / 47.08user 11.82system 1:33.75elapsed 62%CPU (0avgtext+0avgdata 110284maxresident)k 216inputs+0outputs (0major+913424minor)pagefaults 0swaps