{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "honey-understanding",
   "metadata": {},
   "source": [
    "# Tutorial about logging"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "worth-latest",
   "metadata": {},
   "source": [
    "Logging as supplied by the python standard library can be used.\n",
    "\n",
    "We make use of the standard logging levels DEBUG, INFO, WARNING, ERROR, CRITICAL."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "green-western",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import logging\n",
    "import sys\n",
    "from pathlib import Path\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "import locan as lc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "temporal-israeli",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "lc.show_versions(dependencies=False, verbose=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "reduced-mambo",
   "metadata": {},
   "source": [
    "## Activate logging"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "brave-worthy",
   "metadata": {},
   "source": [
    "In any script or notebook logging has to be enabled e.g. for streaming to stdout.\n",
    "\n",
    "For changing the configuration logging has to be reloaded or the kernel be restarted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aboriginal-isaac",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "logging.basicConfig(stream=sys.stdout, level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "impossible-trail",
   "metadata": {},
   "source": [
    "A top-level logger has to be instantiated to process any log messages from the library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "circular-secret",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "logger = logging.getLogger()\n",
    "logger"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "outdoor-birth",
   "metadata": {},
   "source": [
    "Further log messages can be added:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "early-karen",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "logger.info(\"test\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "natural-share",
   "metadata": {},
   "source": [
    "### Handling locan.logger"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bridal-partition",
   "metadata": {},
   "source": [
    "To change the filter level of locan log records, use an instance of the locan logger identified by its module name."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "sunset-break",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "locan_logger = logging.getLogger('locan')\n",
    "locan_logger.setLevel(logging.INFO)\n",
    "locan_logger"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "constant-affect",
   "metadata": {},
   "source": [
    "## Logging in locan"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27dedab3-20df-42cf-95b7-dadfbcc3b791",
   "metadata": {},
   "source": [
    "Many functions provide warnings if some unusual behavior occurs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96d55ad0-8a01-4d35-96b4-8fc50c329d44",
   "metadata": {},
   "outputs": [],
   "source": [
    "locdata = lc.LocData.from_coordinates([(0, 0), (1, 2), (2, 1), (5, 5)])\n",
    "locdata.region = lc.Rectangle((0, 0), 2, 2, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cardiovascular-riverside",
   "metadata": {},
   "source": [
    "Changing the level of the locan logger to `logging.WARNING` or higher, will switch off most locan log records."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "56e8cb19-8451-44f3-92e1-9c7fe3822e37",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "locan_logger.setLevel(logging.ERROR)\n",
    "\n",
    "locdata = lc.LocData.from_coordinates([(0, 0), (1, 2), (2, 1), (5, 5)])\n",
    "locdata.region = lc.Rectangle((0, 0), 2, 2, 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42236de9-d8b4-4ef8-b62a-06a397ea5e6e",
   "metadata": {},
   "outputs": [],
   "source": [
    "locan_logger.setLevel(logging.INFO)\n",
    "\n",
    "locdata = lc.LocData.from_coordinates([(0, 0), (1, 2), (2, 1), (5, 5)])\n",
    "locdata.region = lc.Rectangle((0, 0), 2, 2, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f81a7869-570a-4f60-97c8-4364c7a58f7d",
   "metadata": {},
   "source": [
    "Levels can be set for selected loggers by specifying the module or function name:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ada13c8c-ff17-42bd-b958-3ce033aeab4b",
   "metadata": {},
   "outputs": [],
   "source": [
    "logger_locan_data = logging.getLogger('locan.data')\n",
    "logger_locan_data.setLevel(logging.ERROR)\n",
    "\n",
    "locdata = lc.LocData.from_coordinates([(0, 0), (1, 2), (2, 1), (5, 5)])\n",
    "locdata.region = lc.Rectangle((0, 0), 2, 2, 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1faa6545-ff05-40da-b8ee-05640489443e",
   "metadata": {},
   "outputs": [],
   "source": [
    "logger_locan_data.setLevel(logging.INFO)\n",
    "\n",
    "locdata = lc.LocData.from_coordinates([(0, 0), (1, 2), (2, 1), (5, 5)])\n",
    "locdata.region = lc.Rectangle((0, 0), 2, 2, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "built-marijuana",
   "metadata": {},
   "source": [
    "### Logging in a pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "environmental-provision",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def computation(self, file):\n",
    "    logger.info(f'computation started for file: {file}')\n",
    "    return self"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "vulnerable-enemy",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "pipes = [lc.Pipeline(computation=computation, file=file).compute() for file in range(3)]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sufficient-database",
   "metadata": {},
   "source": [
    "Another example how to use logging in analysis pipelines is given by the `computation_test` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "integral-party",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "pipes = [lc.Pipeline(computation=lc.analysis.pipeline.computation_test, locdata=file).compute() for file in range(3)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "modular-mortgage",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(pipes[0].computation_as_string())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "silver-bruce",
   "metadata": {},
   "source": [
    "### Logging in multiprocessing with ray"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "social-bahamas",
   "metadata": {},
   "source": [
    "To enable logging in multiprocessing using ray you need to include a default configuration in the computation function: `logging.basicConfig(level=logging.INFO)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e9559d61-7126-48ee-a22d-1ea7db538b9c",
   "metadata": {},
   "outputs": [],
   "source": [
    "if False:\n",
    "    def computation(self, file):\n",
    "        logging.basicConfig(level=logging.INFO)\n",
    "        logger.info(f'computation started for file: {file}')\n",
    "        return self"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "981c60a4-c872-4323-ad07-efa7d4cfb36e",
   "metadata": {},
   "outputs": [],
   "source": [
    "if False:\n",
    "    import ray\n",
    "\n",
    "    ray.init()\n",
    "    # ray.init(num_cpus = 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "303f6ce0-8ece-454c-b172-cb000ceacb8a",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "if False:\n",
    "    @ray.remote\n",
    "    def worker(file):\n",
    "        pipe = lc.Pipeline(computation=computation, file=file).compute()\n",
    "        return pipe\n",
    "\n",
    "    futures = [worker.remote(file) for file in range(3)]\n",
    "    pipes = ray.get(futures)\n",
    "    len(pipes)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "nuclear-father",
   "metadata": {},
   "source": [
    "## Logging in locan - third party libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "pleased-heating",
   "metadata": {},
   "source": [
    "Some third-party libraries provide their own logging system. Typically the individual loggers can be imported and modified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "expected-potato",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import trackpy as tr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "contemporary-trailer",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "tr.logger"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "vertical-handy",
   "metadata": {},
   "source": [
    "alternatively"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "arctic-worker",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "trackpy_logger = logging.getLogger('trackpy')\n",
    "trackpy_logger"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "finite-belly",
   "metadata": {},
   "source": [
    "Depending on the library various methods can be used to change the logging level. All of the following can be used."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfe8e146-d1c9-479f-a67a-37fd1d7ac6ba",
   "metadata": {},
   "source": [
    "`trackpy_logger.setLevel(logging.WARN)`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed3418c6-53ae-4db0-94cc-0d9f0c4db2bc",
   "metadata": {},
   "source": [
    "`tr.logger.setLevel(logging.WARN)`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4ab71a9-fa9a-424a-9fea-99ce06dfe7b0",
   "metadata": {},
   "source": [
    "`tr.quiet()`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de3f6358-6b4b-421f-8520-91b2c651bf54",
   "metadata": {},
   "source": [
    "`tr.ignore_logging()`  # this switches off the trackpy logging system and forwards all logs up the logger hirarchy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1f079a5e-9c7d-4dd9-846b-3a050882479e",
   "metadata": {},
   "outputs": [],
   "source": [
    "dat = lc.simulate_tracks(n_walks=1, n_steps=100, ranges=((0,1000),(0,1000)),\n",
    "                      diffusion_constant=1, seed=1)\n",
    "\n",
    "dat.print_meta()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "innocent-proxy",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "locdata_new, track_series = lc.track(dat, search_range=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0169a75a-0ea4-454e-8b90-34a5e4a46ba2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "trackpy_logger.setLevel(logging.WARN)\n",
    "locdata_new, track_series = lc.track(dat, search_range=5)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}