././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1746801363.9563575 proglog-0.1.12/0000755000175100001660000000000015007411324012644 5ustar00runnerdocker././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1746801359.0 proglog-0.1.12/LICENSE0000644000175100001660000000212215007411317013650 0ustar00runnerdockerMIT License Copyright (c) 2017 Edinburgh Genome Foundry, University of Edinburgh Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1746801363.9553573 proglog-0.1.12/PKG-INFO0000644000175100001660000000143215007411324013741 0ustar00runnerdockerMetadata-Version: 2.4 Name: proglog Version: 0.1.12 Summary: Log and progress bar manager for console, notebooks, web... Author: Zulko License-Expression: MIT Project-URL: Homepage, https://github.com/Edinburgh-Genome-Foundry/proglog Keywords: logger,log,progress,bar Description-Content-Type: text/x-rst License-File: LICENSE Requires-Dist: tqdm Dynamic: license-file Proglog ======= Proglog is a progress logging system for Python. It allows to build complex libraries while giving the user control on the management of logs, callbacks and progress bars. Infos ----- **PIP installation:** .. code:: bash pip install proglog **Github Page:** ``_ **License:** MIT Copyright 2017 Edinburgh Genome Foundry, University of Edinburgh ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1746801359.0 proglog-0.1.12/README.rst0000755000175100001660000001672415007411317014352 0ustar00runnerdocker.. raw:: html

Proglog Logo

.. image:: https://github.com/Edinburgh-Genome-Foundry/Proglog/actions/workflows/build.yml/badge.svg :target: https://github.com/Edinburgh-Genome-Foundry/Proglog/actions/workflows/build.yml :alt: GitHub CI build status .. image:: https://coveralls.io/repos/github/Edinburgh-Genome-Foundry/Proglog/badge.svg?branch=master :target: https://coveralls.io/github/Edinburgh-Genome-Foundry/Proglog?branch=master Proglog is a progress logging system for Python. It allows to build complex libraries while giving your users control over logs, callbacks and progress bars. **What problems does it solve ?** Libraries like `tqdm `_ or `progress `_ are great for quickly adding progress bars to your scripts, but become difficult to manage when building larger projects. For instance, you will need to write different code depending on whether you are displaying the progress in a console, a Jupyter notebook, or a webpage. Sometimes a single program may have to handle many logs and progress bars coming from different subprograms and libraries, at which case you may want to let the final user decide which progress bars they want to display or to mute, even when these progress bars are handled deep down in your program. For instance if your program 1 calls a program 2 and program 3 (possibly from other libraries), you may want to be able to silence the progress bars of routine 2, or to only show the progress bars of routine 1. As long as all routines use Proglog, this will be easy to do. .. raw:: html

You may also want to log more than just progress bars, have specific callback fonctions, print the logs in human-readable format... Proglog provides all these features. Usage ----- Assume that you are writing a library called ``my_library`` in which you define a routine as follows: .. code:: python import time # for simulating computing time from proglog import default_bar_logger def my_routine(iterations=10, logger='bar'): """Run several loops to showcase Proglog.""" logger = default_bar_logger(logger) # shorthand to generate a bar logger for i in logger.iter_bar(iteration=range(iterations)): for j in logger.iter_bar(animal=['dog', 'cat', 'rat', 'duck']): time.sleep(0.1) # simulate some computing time Now when the library users run a program in the console, they will get a console progress bar: .. code:: python from my_library import my_routine my_routine() .. raw:: html

If the users run the routine inside a Jupyter/IPython notebook, they only need to write ``proglog.notebook()`` at the beginning of the notebook to obtain HTML progress bars: .. code:: python import proglog proglog.notebook() from my_library import my_routine my_routine() .. raw:: html

If the user wishes to turn off all progress bars: .. code:: python from my_library import my_routine my_routine(logger=None) If the user is running the routine on a web server and would want to attach the data to an asynchronous Python-RQ job, all they need is yet a different logger: .. code:: python from proglog import RqWorkerBarLogger from my_library import my_routine logger = RqWorkerBarLogger(job=some_python_rq_job) my_routine(logger=logger) This allows to then display progress bars on the website such as these (see the `EGF CUBA `_ project for an example of website using Proglog): .. raw:: html

The user may also want a custom progress logger which selectively ignores the ``animals`` progress bar, and only updates its bars every second (to save computing time): .. code:: python from proglog import TqdmProgressBarLogger from my_library import my_routine logger = TqdmProgressBarLogger(ignored_bars=('animal',), min_time_interval=1.0) my_routine(logger=logger) Proglog loggers can be used for much more than just progress bars. They can in fact store any kind of data with a simple API: .. code:: python logger(message='Now running the main program, be patient...') logger(current_animal='cat') logger(last_number_tried=1235) For more complex customization, such as adding callback functions which will be executed every time the logger's state is updated, simply create a new logger class: .. code:: python from proglog import ProgressBarLogger from my_library import my_routine class MyBarLogger(ProgressBarLogger): def callback(self, **changes): # Every time the logger is updated, this function is called with # the `changes` dictionnary of the form `parameter: new value`. for (parameter, new_value) in changes.items(): print ('Parameter %s is now %s' % (parameter, value)) logger = MyBarLogger() my_routine(logger=logger) When writing libraries which all log progress and may depend on each other, simply pass the Proglog logger from one program to its dependencies, to obtain one logger keeping track of all progress across libraries at once: .. raw:: html

Note that this implies that not two libraries use the same variables or loop names, which can be avoided by attributing prefixes to these names: .. code:: python for i in logger.iter_bar(iteration=range(iterations), bar_prefix='libraryname_'): ... Installation ------------ You can install Proglog through PIP: .. code:: shell pip install proglog Alternatively, you can unzip the sources in a folder and type: .. code:: shell python setup.py install To use the ``tqdm`` notebook-style progress bars you need to install iwidgets: .. code:: shell pip install ipywidgets This `should automatically enable it `_; for older versions try: .. code:: shell jupyter nbextension enable --py --sys-prefix widgetsnbextension License = MIT ------------- Proglog is an open-source software originally written at the `Edinburgh Genome Foundry `_ by `Zulko `_ and `released on Github `_ under the MIT license (Copyright 2017 Edinburgh Genome Foundry, University of Edinburgh). Proglog was not written by loggology experts, it *just works* with our projects and we use it a lot. Everyone is welcome to contribute if you find bugs or limitations ! ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1746801363.9533575 proglog-0.1.12/proglog/0000755000175100001660000000000015007411324014315 5ustar00runnerdocker././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1746801359.0 proglog-0.1.12/proglog/__init__.py0000755000175100001660000000044315007411317016434 0ustar00runnerdocker"""geneblocks/__init__.py""" # __all__ = [] from .proglog import ( ProgressLogger, ProgressBarLogger, TqdmProgressBarLogger, notebook, RqWorkerProgressLogger, RqWorkerBarLogger, MuteProgressBarLogger, default_bar_logger, ) from .version import __version__ ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1746801359.0 proglog-0.1.12/proglog/proglog.py0000644000175100001660000003314615007411317016351 0ustar00runnerdocker"""Implements the generic progress logger class, and the ProgressBar class.""" from tqdm import tqdm, tqdm_notebook from collections import OrderedDict import time SETTINGS = {"notebook": False} def notebook(turn="on"): SETTINGS["notebook"] = True if (turn == "on") else False def troncate_string(s, max_length=25): return s if (len(s) < max_length) else (s[:max_length] + "...") class ProgressLogger: """Generic class for progress loggers. A progress logger contains a "state" dictionary. Parameters ---------- init_state : dict Dictionary representing the initial state. """ def __init__(self, init_state=None): self.state = {} self.stored = {} self.logs = [] self.log_indent = 0 if init_state is not None: self.state.update(init_state) def log(self, message): self.logs.append((" " * self.log_indent) + message) def dump_logs(self, filepath=None): if filepath is not None: with open(filepath, "a") as f: f.write("\n".join(self.logs)) else: return "\n".join(self.logs) def callback(self, **kw): """Execute something after the state has been updated by the given state elements. This default callback does nothing, overwrite it by subclassing. """ pass def store(self, **kw): """Store objects in the logger and trigger ``self.store_callback``. This works exactly like ``logger()``, but the later is meant for simple data objects (text, numbers) that will be sent over the network or written to a file. The ``store`` method expects rather large objects which are not necessarily serializable, and will be used eg to draw plots on the fly. """ self.stored.update(kw) self.store_callback(**kw) def store_callback(self, **kw): """Execute something after the store has been updated by the given state elements. This default callback does nothing, overwrite it by subclassing """ pass def iter(self, **kw): """Iterate through a list while updating the state. Examples -------- >>> for username in logger.iter(user=['tom', 'tim', 'lea']): >>> # At every loop, logger.state['user'] is updated >>> print(username) """ for field, iterable in kw.items(): for it in iterable: self(**{field: it}) yield it def __call__(self, **kw): self.state.update(kw) self.callback(**kw) class ProgressBarLogger(ProgressLogger): """Generic class for progress loggers. A progress logger contains a "state" dictionary. Parameters ---------- init_state : dict Initial state of the logger. bars : None, list, tuple, or dict, optional Either None (will be initialized with no bar) or a list/tuple of bar names (e.g., ['main', 'sub']) which will be initialized with index -1 and no total, or a dictionary (possibly ordered) of bars, of the form `{bar_1: {title: 'bar1', index: 2, total: 23}, bar_2: {...}}`. ignored_bars : None, list of str, or 'all_others', optional Either None (newly met bars will be added) or a list of blacklisted bar names, or 'all_others' to signify that all bar names not already in `self.bars` will be ignored. logged_bars min_time_interval : int or float Time in seconds between progress bar updates. ignore_bars_under : int """ bar_indent = 2 def __init__( self, init_state=None, bars=None, ignored_bars=None, logged_bars="all", min_time_interval=0, ignore_bars_under=0, ): ProgressLogger.__init__(self, init_state) if bars is None: bars = OrderedDict() elif isinstance(bars, (list, tuple)): bars = OrderedDict( [ (b, dict(title=b, index=-1, total=None, message=None, indent=0)) for b in bars ] ) if isinstance(ignored_bars, (list, tuple)): ignored_bars = set(ignored_bars) self.ignored_bars = ignored_bars self.logged_bars = logged_bars self.state["bars"] = bars self.min_time_interval = min_time_interval self.ignore_bars_under = ignore_bars_under @property def bars(self): """Return ``self.state['bars'].``""" return self.state["bars"] def bar_is_ignored(self, bar): if self.ignored_bars is None: return False elif self.ignored_bars == "all_others": return bar not in self.bars else: return bar in self.ignored_bars def bar_is_logged(self, bar): if not self.logged_bars: return False elif self.logged_bars == "all": return True else: return bar in self.logged_bars def iterable_is_too_short(self, iterable): length = len(iterable) if hasattr(iterable, "__len__") else None return (length is not None) and (length < self.ignore_bars_under) def iter_bar(self, bar_prefix="", **kw): """Iterate through a list while updating a state bar. Parameters ---------- bar_prefix : str Bar prefix. Examples -------- >>> for username in logger.iter_bar(user=['tom', 'tim', 'lea']): >>> # At every loop, logger.state['bars']['user'] is updated >>> # to {index: i, total: 3, title:'user'} >>> print (username) """ if "bar_message" in kw: bar_message = kw.pop("bar_message") else: bar_message = None bar, iterable = kw.popitem() if self.bar_is_ignored(bar) or self.iterable_is_too_short(iterable): return iterable bar = bar_prefix + bar if hasattr(iterable, "__len__"): self(**{bar + "__total": len(iterable)}) def new_iterable(): last_time = time.time() i = 0 # necessary in case the iterator is empty for i, it in enumerate(iterable): now_time = time.time() if (i == 0) or (now_time - last_time > self.min_time_interval): if bar_message is not None: self(**{bar + "__message": bar_message(it)}) self(**{bar + "__index": i}) last_time = now_time yield it if self.bars[bar]["index"] != i: self(**{bar + "__index": i}) self(**{bar + "__index": i + 1}) return new_iterable() def bars_callback(self, bar, attr, value, old_value=None): """Execute a custom action after the progress bars are updated. Parameters ---------- bar Name/ID of the bar to be modified. attr Attribute of the bar attribute to be modified value New value of the attribute old_value Previous value of this bar's attribute. This default callback does nothing, overwrite it by subclassing. """ pass def __call__(self, **kw): items = sorted(kw.items(), key=lambda kv: not kv[0].endswith("total")) for key, value in items: if "__" in key: bar, attr = key.split("__") if self.bar_is_ignored(bar): continue kw.pop(key) if bar not in self.bars: self.bars[bar] = dict(title=bar, index=-1, total=None, message=None) old_value = self.bars[bar][attr] if self.bar_is_logged(bar): new_bar = (attr == "index") and (value < old_value) if (attr == "total") or (new_bar): self.bars[bar]["indent"] = self.log_indent else: self.log_indent = self.bars[bar]["indent"] self.log("[%s] %s: %s" % (bar, attr, value)) self.log_indent += self.bar_indent self.bars[bar][attr] = value self.bars_callback(bar, attr, value, old_value) self.state.update(kw) self.callback(**kw) class TqdmProgressBarLogger(ProgressBarLogger): """Tqdm-powered progress bar for console or Notebooks. Parameters ---------- init_state : dict Initial state of the logger. bars : None, list, tuple, or dict, optional Either None (will be initialized with no bar) or a list/tuple of bar names (e.g., ['main', 'sub']) which will be initialized with index -1 and no total, or a dictionary (possibly ordered) of bars, of the form `{bar_1: {title: 'bar1', index: 2, total: 23}, bar_2: {...}}`. leave_bars : bool, optional Whether to leave the progress bars displayed upon completion. ignored_bars : None, list of str, or 'all_others', optional Either None (newly met bars will be added) or a list of blacklisted bar names, or 'all_others' to signify that all bar names not already in `self.bars` will be ignored. notebook : bool, optional True will make the bars look nice (HTML) in the Jupyter notebook. It is advised to leave to 'default' as the default can be globally set from inside a notebook with `import proglog; proglog.notebook_mode()`. print_messages : bool If True, every `logger(message='something')` will print a message in the console or notebook. min_time_interval : int or float Time in seconds between progress bar updates. ignore_bars_under : int """ def __init__( self, init_state=None, bars=None, leave_bars=False, ignored_bars=None, logged_bars="all", notebook="default", print_messages=True, min_time_interval=0, ignore_bars_under=0, ): ProgressBarLogger.__init__( self, init_state=init_state, bars=bars, ignored_bars=ignored_bars, logged_bars=logged_bars, ignore_bars_under=ignore_bars_under, min_time_interval=min_time_interval, ) self.leave_bars = leave_bars self.tqdm_bars = OrderedDict([(bar, None) for bar in self.bars]) if notebook == "default": notebook = SETTINGS["notebook"] self.notebook = notebook self.print_messages = print_messages self.tqdm = tqdm_notebook if self.notebook else tqdm def new_tqdm_bar(self, bar): """Create a new tqdm bar, possibly replacing an existing one.""" if (bar in self.tqdm_bars) and (self.tqdm_bars[bar] is not None): self.close_tqdm_bar(bar) infos = self.bars[bar] self.tqdm_bars[bar] = self.tqdm( total=infos["total"], desc=infos["title"], postfix=dict(now=troncate_string(str(infos["message"]))), leave=self.leave_bars, ) def close_tqdm_bar(self, bar): """Close and erase the tqdm bar.""" self.tqdm_bars[bar].close() if not self.notebook: self.tqdm_bars[bar] = None def bars_callback(self, bar, attr, value, old_value): if (bar not in self.tqdm_bars) or (self.tqdm_bars[bar] is None): self.new_tqdm_bar(bar) if attr == "index": if value >= old_value: total = self.bars[bar]["total"] if total and (value >= total): self.close_tqdm_bar(bar) else: self.tqdm_bars[bar].update(value - old_value) else: self.new_tqdm_bar(bar) self.tqdm_bars[bar].update(value + 1) elif attr == "message": self.tqdm_bars[bar].set_postfix(now=troncate_string(str(value))) self.tqdm_bars[bar].update(0) def callback(self, **kw): if self.print_messages and ("message" in kw) and kw["message"]: if self.notebook: print(kw["message"]) else: self.tqdm.write(kw["message"]) class RqWorkerProgressLogger: def __init__(self, job): self.job = job if "progress_data" not in self.job.meta: self.job.meta["progress_data"] = {} self.job.save() def callback(self, **kw): self.job.meta["progress_data"] = self.state self.job.save() class RqWorkerBarLogger(RqWorkerProgressLogger, ProgressBarLogger): def __init__( self, job, init_state=None, bars=None, ignored_bars=(), logged_bars="all", min_time_interval=0, ): RqWorkerProgressLogger.__init__(self, job) ProgressBarLogger.__init__( self, init_state=init_state, bars=bars, ignored_bars=ignored_bars, logged_bars=logged_bars, min_time_interval=min_time_interval, ) class MuteProgressBarLogger(ProgressBarLogger): def bar_is_ignored(self, bar): return True def default_bar_logger( logger, bars=None, ignored_bars=None, logged_bars="all", min_time_interval=0, ignore_bars_under=0, ): if logger == "bar": return TqdmProgressBarLogger( bars=bars, ignored_bars=ignored_bars, logged_bars=logged_bars, min_time_interval=min_time_interval, ignore_bars_under=ignore_bars_under, ) elif logger is None: return MuteProgressBarLogger() else: return logger ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1746801359.0 proglog-0.1.12/proglog/version.py0000755000175100001660000000002715007411317016360 0ustar00runnerdocker__version__ = "0.1.12" ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1746801363.9553573 proglog-0.1.12/proglog.egg-info/0000755000175100001660000000000015007411324016007 5ustar00runnerdocker././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1746801363.0 proglog-0.1.12/proglog.egg-info/PKG-INFO0000644000175100001660000000143215007411323017103 0ustar00runnerdockerMetadata-Version: 2.4 Name: proglog Version: 0.1.12 Summary: Log and progress bar manager for console, notebooks, web... Author: Zulko License-Expression: MIT Project-URL: Homepage, https://github.com/Edinburgh-Genome-Foundry/proglog Keywords: logger,log,progress,bar Description-Content-Type: text/x-rst License-File: LICENSE Requires-Dist: tqdm Dynamic: license-file Proglog ======= Proglog is a progress logging system for Python. It allows to build complex libraries while giving the user control on the management of logs, callbacks and progress bars. Infos ----- **PIP installation:** .. code:: bash pip install proglog **Github Page:** ``_ **License:** MIT Copyright 2017 Edinburgh Genome Foundry, University of Edinburgh ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1746801363.0 proglog-0.1.12/proglog.egg-info/SOURCES.txt0000644000175100001660000000043215007411323017671 0ustar00runnerdockerLICENSE README.rst pypi-readme.rst pyproject.toml proglog/__init__.py proglog/proglog.py proglog/version.py proglog.egg-info/PKG-INFO proglog.egg-info/SOURCES.txt proglog.egg-info/dependency_links.txt proglog.egg-info/requires.txt proglog.egg-info/top_level.txt tests/test_basics.py././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1746801363.0 proglog-0.1.12/proglog.egg-info/dependency_links.txt0000644000175100001660000000000115007411323022054 0ustar00runnerdocker ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1746801363.0 proglog-0.1.12/proglog.egg-info/requires.txt0000644000175100001660000000000515007411323020401 0ustar00runnerdockertqdm ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1746801363.0 proglog-0.1.12/proglog.egg-info/top_level.txt0000644000175100001660000000002415007411323020534 0ustar00runnerdockerdist images proglog ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1746801359.0 proglog-0.1.12/pypi-readme.rst0000644000175100001660000000065015007411317015615 0ustar00runnerdockerProglog ======= Proglog is a progress logging system for Python. It allows to build complex libraries while giving the user control on the management of logs, callbacks and progress bars. Infos ----- **PIP installation:** .. code:: bash pip install proglog **Github Page:** ``_ **License:** MIT Copyright 2017 Edinburgh Genome Foundry, University of Edinburgh ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1746801359.0 proglog-0.1.12/pyproject.toml0000644000175100001660000000077315007411317015571 0ustar00runnerdocker[project] name = "proglog" version = "0.1.12" license = "MIT" authors = [{ name = "Zulko" }] description = "Log and progress bar manager for console, notebooks, web..." readme = "pypi-readme.rst" keywords = ["logger", "log", "progress", "bar"] dependencies = ["tqdm"] [project.urls] Homepage = "https://github.com/Edinburgh-Genome-Foundry/proglog" [build-system] requires = ["setuptools"] build-backend = "setuptools.build_meta" [tool.setuptools.packages.find] exclude = ["docs*", "examples*", "tests*"] ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1746801363.9563575 proglog-0.1.12/setup.cfg0000644000175100001660000000004615007411324014465 0ustar00runnerdocker[egg_info] tag_build = tag_date = 0 ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1746801363.9553573 proglog-0.1.12/tests/0000755000175100001660000000000015007411324014006 5ustar00runnerdocker././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1746801359.0 proglog-0.1.12/tests/test_basics.py0000644000175100001660000000123615007411317016667 0ustar00runnerdockerimport time # for simulating computing time from proglog import default_bar_logger from proglog import TqdmProgressBarLogger def my_routine(iterations=10, logger="bar"): """Run several loops to showcase Proglog.""" logger = default_bar_logger(logger) # shorthand to generate a bar logger for i in logger.iter_bar(iteration=range(iterations)): for j in logger.iter_bar(animal=["dog", "cat", "rat", "duck"]): time.sleep(0.02) # simulate some computing time def test_basic_log(): my_routine() def test_tqdm_log(): logger = TqdmProgressBarLogger(ignored_bars=("animal",), min_time_interval=1.0) my_routine(logger=logger)