././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987214.9257667 linkify_it_py-2.2.0/0000755000175100017510000000000015244502417014036 5ustar00runnerrunner././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987211.2088614 linkify_it_py-2.2.0/CHANGELOG.md0000644000175100017510000000611615244502413015647 0ustar00runnerrunner# Change Log ## v2.2.0 (2026-08-29) - Fix astral-plane Unicode handling in links, including host names and link terminators ([#87](https://github.com/tsutsu3/linkify-it-py/pull/87)) - Drop the runtime dependency on `uc-micro-py` ([#87](https://github.com/tsutsu3/linkify-it-py/pull/87)) ## v2.1.1 (2026-08-25) Security release: `LinkifyIt.match()` was quadratic on untrusted input when linkify is enabled ([GHSA-8m2q-wq3r-6hq8](https://github.com/tsutsu3/linkify-it-py/security/advisories/GHSA-8m2q-wq3r-6hq8)). - Fix quadratic complexity in `LinkifyIt.match()`, port of linkify-it 5.0.1 and 5.0.2 (upstream [CVE-2026-48801](https://github.com/advisories/GHSA-22p9-wv53-3rq4), [CVE-2026-59887](https://github.com/advisories/GHSA-v245-v573-v5vm)) ([#82](https://github.com/tsutsu3/linkify-it-py/pull/82)) - Match validators at a position instead of slicing the tail, needed because Python 3.10 does not optimize the `^` anchor ([#82](https://github.com/tsutsu3/linkify-it-py/pull/82)) - Allow `;` in the email name, matching linkify-it. **Behavior change**: `a;b@example.com` is now linkified ([#82](https://github.com/tsutsu3/linkify-it-py/pull/82)) - Update port.yml (linkify-it v5.0.2) ([#82](https://github.com/tsutsu3/linkify-it-py/pull/82)) ## v2.1.0 (2026-03-01) - Drop support for EOL Python 3.7, 3.8 ([bd78752](https://github.com/tsutsu3/linkify-it-py/commit/bd78752)) - Drop support for EOL Python 3.9 ([#75](https://github.com/tsutsu3/linkify-it-py/pull/75)) by [@hugovk](https://github.com/hugovk) - Add support for Python 3.14 ([#75](https://github.com/tsutsu3/linkify-it-py/pull/75)) by [@hugovk](https://github.com/hugovk) - Migrate to OIDC trusted publishing for PyPI and TestPyPI ([#77](https://github.com/tsutsu3/linkify-it-py/pull/77)) - Add workflow permissions for security hardening ([#77](https://github.com/tsutsu3/linkify-it-py/pull/77)) - Bump GitHub Actions dependencies ([#71](https://github.com/tsutsu3/linkify-it-py/pull/71), [#73](https://github.com/tsutsu3/linkify-it-py/pull/73)) ## v2.0.3 (2024-02-04) - Update port.yml (linkify-it v5.0.0) ([#54](https://github.com/tsutsu3/linkify-it-py/pull/54)) - Fix rtd ([#52](https://github.com/tsutsu3/linkify-it-py/pull/52)) - Add linkify-it-py-demo url ([#51](https://github.com/tsutsu3/linkify-it-py/pull/51)) - Fix package build ([#49](https://github.com/tsutsu3/linkify-it-py/pull/49)) ## v2.0.2 (2023-05-02) - Fix missing files to the test ([#44](https://github.com/tsutsu3/linkify-it-py/pull/44)) ## v2.0.1 (2023-05-02) - Update development tools - Fix sdist is missing tests ## v2.0.0 (2022-05-07) - Add `matchAtStart` method to match full URLs at the start of the string. - Fixed paired symbols (`()`, `{}`, `""`, etc.) after punctuation. - `---` option now affects parsing of emails (e.g. `user@example.com---`) ## v1.0.3 (2021-12-18) - Fixed [#98](https://github.com/markdown-it/linkify-it/issues/98). Don't count `;` at the end of link (when followed with space). ## v1.0.2 (2021-10-09) - Fix: Schema key containing - not producing matches (#26) ## v1.0.1 (2020-12-18) - Add manifest - Add codecov.yml ## v1.0.0 (2020-11-15) - First release ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987211.2088614 linkify_it_py-2.2.0/LICENSE0000644000175100017510000000427215244502413015044 0ustar00runnerrunnerMIT License Copyright (c) 2020 tsutsu3 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. =============================================================================== Original Javascript version: Copyright (c) 2015 Vitaly Puzrin. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987211.2088614 linkify_it_py-2.2.0/MANIFEST.in0000644000175100017510000000010715244502413015566 0ustar00runnerrunnerinclude CHANGELOG.md include tox.ini recursive-include test *.txt *.py ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987214.9257257 linkify_it_py-2.2.0/PKG-INFO0000644000175100017510000002055115244502417015136 0ustar00runnerrunnerMetadata-Version: 2.4 Name: linkify-it-py Version: 2.2.0 Summary: Links recognition library with FULL unicode support. Author: tsutsu3 License: MIT Project-URL: Homepage, https://github.com/tsutsu3/linkify-it-py Keywords: linkify,linkifier,autolink,autolinker Classifier: Development Status :: 5 - Production/Stable Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.10 Classifier: Programming Language :: Python :: 3.11 Classifier: Programming Language :: Python :: 3.12 Classifier: Programming Language :: Python :: 3.13 Classifier: Programming Language :: Python :: 3.14 Classifier: License :: OSI Approved :: MIT License Classifier: Operating System :: OS Independent Classifier: Intended Audience :: Developers Classifier: Topic :: Software Development :: Libraries :: Python Modules Requires-Python: >=3.10 Description-Content-Type: text/markdown License-File: LICENSE Provides-Extra: test Requires-Dist: pytest; extra == "test" Requires-Dist: coverage; extra == "test" Requires-Dist: pytest-cov; extra == "test" Requires-Dist: pytest-timeout; extra == "test" Provides-Extra: dev Requires-Dist: pre-commit; extra == "dev" Requires-Dist: isort; extra == "dev" Requires-Dist: flake8; extra == "dev" Requires-Dist: black; extra == "dev" Requires-Dist: pyproject-flake8; extra == "dev" Provides-Extra: benchmark Requires-Dist: pytest; extra == "benchmark" Requires-Dist: pytest-benchmark; extra == "benchmark" Provides-Extra: doc Requires-Dist: sphinx; extra == "doc" Requires-Dist: sphinx_book_theme; extra == "doc" Requires-Dist: myst-parser; extra == "doc" Dynamic: license-file # linkify-it-py [![CI](https://github.com/tsutsu3/linkify-it-py/actions/workflows/github-ci.yml/badge.svg)](https://github.com/tsutsu3/linkify-it-py/actions/workflows/github-ci.yml) [![pypi](https://img.shields.io/pypi/v/linkify-it-py)](https://pypi.org/project/linkify-it-py/) [![Anaconda-Server Badge](https://anaconda.org/conda-forge/linkify-it-py/badges/version.svg)](https://anaconda.org/conda-forge/linkify-it-py) [![Documentation Status](https://readthedocs.org/projects/linkify-it-py/badge/?version=latest)](https://linkify-it-py.readthedocs.io/en/latest/?badge=latest) [![codecov](https://codecov.io/gh/tsutsu3/linkify-it-py/branch/main/graph/badge.svg)](https://codecov.io/gh/tsutsu3/linkify-it-py) This is Python port of [linkify-it](https://github.com/markdown-it/linkify-it). > Links recognition library with FULL unicode support. > Focused on high quality link patterns detection in plain text. __[Demo](https://linkify-it-py-demo.vercel.app/)__ __[Javascript Demo](http://markdown-it.github.io/linkify-it/)__ Why it's awesome: - Full unicode support, _with astral characters_! - International domains support. - Allows rules extension & custom normalizers. ## Python Version Support Tested on Python 3.10–3.14. The primary version for code coverage follows the latest [security phase](https://devguide.python.org/versions/) release (currently __3.12__). ## Install ```bash pip install linkify-it-py ``` or ```bash conda install -c conda-forge linkify-it-py ``` ## Usage examples ### Example 1. Simple use ```python from linkify_it import LinkifyIt linkify = LinkifyIt() print(linkify.test("Site github.com!")) # => True print(linkify.match("Site github.com!")) # => [linkify_it.main.Match({ # 'schema': '', # 'index': 5, # 'last_index': 15, # 'raw': 'github.com', # 'text': 'github.com', # 'url': 'http://github.com' # }] ``` ### Example 2. With options ```python from linkify_it import LinkifyIt from linkify_it.tlds import TLDS # Reload full tlds list & add unofficial `.onion` domain. linkify = ( LinkifyIt() .tlds(TLDS) # Reload with full tlds list .tlds("onion", True) # Add unofficial `.onion` domain .add("git:", "http:") # Add `git:` protocol as "alias" .add("ftp:", None) # Disable `ftp:` protocol .set({"fuzzy_ip": True}) # Enable IPs in fuzzy links (without schema) ) print(linkify.test("Site tamanegi.onion!")) # => True print(linkify.match("Site tamanegi.onion!")) # => [linkify_it.main.Match({ # 'schema': '', # 'index': 5, # 'last_index': 19, # 'raw': 'tamanegi.onion', # 'text': 'tamanegi.onion', # 'url': 'http://tamanegi.onion' # }] ``` ### Example 3. Add twitter mentions handler ```python from linkify_it import LinkifyIt linkify = LinkifyIt() def validate(obj, text, pos): tail = text[pos:] if not obj.re.get("twitter"): obj.re["twitter"] = re.compile( "^([a-zA-Z0-9_]){1,15}(?!_)(?=$|" + obj.re["src_ZPCc"] + ")" ) if obj.re["twitter"].search(tail): if pos > 2 and tail[pos - 2] == "@": return False return len(obj.re["twitter"].search(tail).group()) return 0 def normalize(obj, match): match.url = "https://twitter.com/" + re.sub(r"^@", "", match.url) linkify.add("@", {"validate": validate, "normalize": normalize}) ``` ## API [API documentation](https://linkify-it-py.readthedocs.io/en/latest/) ### LinkifyIt(schemas, options) Creates new linkifier instance with optional additional schemas. By default understands: - `http(s)://...` , `ftp://...`, `mailto:...` & `//...` links - "fuzzy" links and emails (google.com, foo@bar.com). `schemas` is an dict, where each key/value describes protocol/rule: - __key__ - link prefix (usually, protocol name with `:` at the end, `skype:` for example). `linkify-it-py` makes sure that prefix is not preceded with alphanumeric char. - __value__ - rule to check tail after link prefix - _str_ - just alias to existing rule - _dict_ - _validate_ - either a `re.Pattern` (start with `^`, and don't include the link prefix itself), or a validator `function` which, given arguments _self_, _text_ and _pos_, returns the length of a match in _text_ starting at index _pos_. _pos_ is the index right after the link prefix. _self_ can be used to access the linkify object to cache data. - _normalize_ - optional function to normalize text & url of matched result (for example, for twitter mentions). `options`: - __fuzzy_link__ - recognize URL-s without `http(s)://` head. Default `True`. - __fuzzy_ip__ - allow IPs in fuzzy links above. Can conflict with some texts like version numbers. Default `False`. - __fuzzy_email__ - recognize emails without `mailto:` prefix. Default `True`. - __---__ - set `True` to terminate link with `---` (if it's considered as long dash). ### .test(text) Searches linkifiable pattern and returns `True` on success or `False` on fail. ### .pretest(text) Quick check if link MAY BE can exist. Can be used to optimize more expensive `.test()` calls. Return `False` if link can not be found, `True` - if `.test()` call needed to know exactly. ### .test_schema_at(text, name, position) Similar to `.test()` but checks only specific protocol tail exactly at given position. Returns length of found pattern (0 on fail). ### .match(text) Returns `list` of found link matches or null if nothing found. Each match has: - __schema__ - link schema, can be empty for fuzzy links, or `//` for protocol-neutral links. - __index__ - offset of matched text - __last_index__ - index of next char after mathch end - __raw__ - matched text - __text__ - normalized text - __url__ - link, generated from matched text ### .matchAtStart(text) Checks if a match exists at the start of the string. Returns `Match` (see docs for `match(text)`) or null if no URL is at the start. Doesn't work with fuzzy links. ### .tlds(list_tlds, keep_old=False) Load (or merge) new tlds list. Those are needed for fuzzy links (without schema) to avoid false positives. By default: - 2-letter root zones are ok. - biz|com|edu|gov|net|org|pro|web|xxx|aero|asia|coop|info|museum|name|shop|рф are ok. - encoded (`xn--...`) root zones are ok. If that's not enough, you can reload defaults with more detailed zones list. ### .add(key, value) Add a new schema to the schemas object. As described in the constructor definition, `key` is a link prefix (`skype:`, for example), and `value` is a `str` to alias to another schema, or an `dict` with `validate` and optionally `normalize` definitions. To disable an existing rule, use `.add(key, None)`. ### .set(options) Override default options. Missed properties will not be changed. ## License [MIT](https://github.com/tsutsu3/linkify-it-py/blob/master/LICENSE) ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987211.2088614 linkify_it_py-2.2.0/README.md0000644000175100017510000001541315244502413015315 0ustar00runnerrunner# linkify-it-py [![CI](https://github.com/tsutsu3/linkify-it-py/actions/workflows/github-ci.yml/badge.svg)](https://github.com/tsutsu3/linkify-it-py/actions/workflows/github-ci.yml) [![pypi](https://img.shields.io/pypi/v/linkify-it-py)](https://pypi.org/project/linkify-it-py/) [![Anaconda-Server Badge](https://anaconda.org/conda-forge/linkify-it-py/badges/version.svg)](https://anaconda.org/conda-forge/linkify-it-py) [![Documentation Status](https://readthedocs.org/projects/linkify-it-py/badge/?version=latest)](https://linkify-it-py.readthedocs.io/en/latest/?badge=latest) [![codecov](https://codecov.io/gh/tsutsu3/linkify-it-py/branch/main/graph/badge.svg)](https://codecov.io/gh/tsutsu3/linkify-it-py) This is Python port of [linkify-it](https://github.com/markdown-it/linkify-it). > Links recognition library with FULL unicode support. > Focused on high quality link patterns detection in plain text. __[Demo](https://linkify-it-py-demo.vercel.app/)__ __[Javascript Demo](http://markdown-it.github.io/linkify-it/)__ Why it's awesome: - Full unicode support, _with astral characters_! - International domains support. - Allows rules extension & custom normalizers. ## Python Version Support Tested on Python 3.10–3.14. The primary version for code coverage follows the latest [security phase](https://devguide.python.org/versions/) release (currently __3.12__). ## Install ```bash pip install linkify-it-py ``` or ```bash conda install -c conda-forge linkify-it-py ``` ## Usage examples ### Example 1. Simple use ```python from linkify_it import LinkifyIt linkify = LinkifyIt() print(linkify.test("Site github.com!")) # => True print(linkify.match("Site github.com!")) # => [linkify_it.main.Match({ # 'schema': '', # 'index': 5, # 'last_index': 15, # 'raw': 'github.com', # 'text': 'github.com', # 'url': 'http://github.com' # }] ``` ### Example 2. With options ```python from linkify_it import LinkifyIt from linkify_it.tlds import TLDS # Reload full tlds list & add unofficial `.onion` domain. linkify = ( LinkifyIt() .tlds(TLDS) # Reload with full tlds list .tlds("onion", True) # Add unofficial `.onion` domain .add("git:", "http:") # Add `git:` protocol as "alias" .add("ftp:", None) # Disable `ftp:` protocol .set({"fuzzy_ip": True}) # Enable IPs in fuzzy links (without schema) ) print(linkify.test("Site tamanegi.onion!")) # => True print(linkify.match("Site tamanegi.onion!")) # => [linkify_it.main.Match({ # 'schema': '', # 'index': 5, # 'last_index': 19, # 'raw': 'tamanegi.onion', # 'text': 'tamanegi.onion', # 'url': 'http://tamanegi.onion' # }] ``` ### Example 3. Add twitter mentions handler ```python from linkify_it import LinkifyIt linkify = LinkifyIt() def validate(obj, text, pos): tail = text[pos:] if not obj.re.get("twitter"): obj.re["twitter"] = re.compile( "^([a-zA-Z0-9_]){1,15}(?!_)(?=$|" + obj.re["src_ZPCc"] + ")" ) if obj.re["twitter"].search(tail): if pos > 2 and tail[pos - 2] == "@": return False return len(obj.re["twitter"].search(tail).group()) return 0 def normalize(obj, match): match.url = "https://twitter.com/" + re.sub(r"^@", "", match.url) linkify.add("@", {"validate": validate, "normalize": normalize}) ``` ## API [API documentation](https://linkify-it-py.readthedocs.io/en/latest/) ### LinkifyIt(schemas, options) Creates new linkifier instance with optional additional schemas. By default understands: - `http(s)://...` , `ftp://...`, `mailto:...` & `//...` links - "fuzzy" links and emails (google.com, foo@bar.com). `schemas` is an dict, where each key/value describes protocol/rule: - __key__ - link prefix (usually, protocol name with `:` at the end, `skype:` for example). `linkify-it-py` makes sure that prefix is not preceded with alphanumeric char. - __value__ - rule to check tail after link prefix - _str_ - just alias to existing rule - _dict_ - _validate_ - either a `re.Pattern` (start with `^`, and don't include the link prefix itself), or a validator `function` which, given arguments _self_, _text_ and _pos_, returns the length of a match in _text_ starting at index _pos_. _pos_ is the index right after the link prefix. _self_ can be used to access the linkify object to cache data. - _normalize_ - optional function to normalize text & url of matched result (for example, for twitter mentions). `options`: - __fuzzy_link__ - recognize URL-s without `http(s)://` head. Default `True`. - __fuzzy_ip__ - allow IPs in fuzzy links above. Can conflict with some texts like version numbers. Default `False`. - __fuzzy_email__ - recognize emails without `mailto:` prefix. Default `True`. - __---__ - set `True` to terminate link with `---` (if it's considered as long dash). ### .test(text) Searches linkifiable pattern and returns `True` on success or `False` on fail. ### .pretest(text) Quick check if link MAY BE can exist. Can be used to optimize more expensive `.test()` calls. Return `False` if link can not be found, `True` - if `.test()` call needed to know exactly. ### .test_schema_at(text, name, position) Similar to `.test()` but checks only specific protocol tail exactly at given position. Returns length of found pattern (0 on fail). ### .match(text) Returns `list` of found link matches or null if nothing found. Each match has: - __schema__ - link schema, can be empty for fuzzy links, or `//` for protocol-neutral links. - __index__ - offset of matched text - __last_index__ - index of next char after mathch end - __raw__ - matched text - __text__ - normalized text - __url__ - link, generated from matched text ### .matchAtStart(text) Checks if a match exists at the start of the string. Returns `Match` (see docs for `match(text)`) or null if no URL is at the start. Doesn't work with fuzzy links. ### .tlds(list_tlds, keep_old=False) Load (or merge) new tlds list. Those are needed for fuzzy links (without schema) to avoid false positives. By default: - 2-letter root zones are ok. - biz|com|edu|gov|net|org|pro|web|xxx|aero|asia|coop|info|museum|name|shop|рф are ok. - encoded (`xn--...`) root zones are ok. If that's not enough, you can reload defaults with more detailed zones list. ### .add(key, value) Add a new schema to the schemas object. As described in the constructor definition, `key` is a link prefix (`skype:`, for example), and `value` is a `str` to alias to another schema, or an `dict` with `validate` and optionally `normalize` definitions. To disable an existing rule, use `.add(key, None)`. ### .set(options) Override default options. Missed properties will not be changed. ## License [MIT](https://github.com/tsutsu3/linkify-it-py/blob/master/LICENSE) ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987214.9222648 linkify_it_py-2.2.0/linkify_it/0000755000175100017510000000000015244502417016177 5ustar00runnerrunner././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987211.2108793 linkify_it_py-2.2.0/linkify_it/__init__.py0000644000175100017510000000015715244502413020307 0ustar00runnerrunnerfrom .main import LinkifyIt # noqa: F401p from .main import SchemaError # noqa: F401p __version__ = "2.2.0" ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987211.2111893 linkify_it_py-2.2.0/linkify_it/_ucre_data.py0000644000175100017510000000616715244502413020645 0ustar00runnerrunner"""Unicode character classes for :mod:`linkify_it.ucre`. Generated by ``support/generate_ucre.py`` from Unicode 16.0.0. Do not edit by hand. """ UNIDATA_VERSION = "16.0.0" # Match any single character, including newlines. SRC_ANY = "[\\s\\S]" # General_Category=Cc SRC_CC = "[\x00-\x1f\x7f-\x9f]" # noqa: E501 # General_Category=Cf SRC_CF = "[\xad\u0600-\u0605\u061c\u06dd\u070f\u0890-\u0891\u08e2\u180e\u200b-\u200f\u202a-\u202e\u2060-\u2064\u2066-\u206f\ufeff\ufff9-\ufffb\U000110bd\U000110cd\U00013430-\U0001343f\U0001bca0-\U0001bca3\U0001d173-\U0001d17a\U000e0001\U000e0020-\U000e007f]" # noqa: E501 # General_Category=P SRC_P = "[!-\\#%-\\*,-/:-;\\?-@\\[-\\]_\\{\\}\xa1\xa7\xab\xb6-\xb7\xbb\xbf\u037e\u0387\u055a-\u055f\u0589-\u058a\u05be\u05c0\u05c3\u05c6\u05f3-\u05f4\u0609-\u060a\u060c-\u060d\u061b\u061d-\u061f\u066a-\u066d\u06d4\u0700-\u070d\u07f7-\u07f9\u0830-\u083e\u085e\u0964-\u0965\u0970\u09fd\u0a76\u0af0\u0c77\u0c84\u0df4\u0e4f\u0e5a-\u0e5b\u0f04-\u0f12\u0f14\u0f3a-\u0f3d\u0f85\u0fd0-\u0fd4\u0fd9-\u0fda\u104a-\u104f\u10fb\u1360-\u1368\u1400\u166e\u169b-\u169c\u16eb-\u16ed\u1735-\u1736\u17d4-\u17d6\u17d8-\u17da\u1800-\u180a\u1944-\u1945\u1a1e-\u1a1f\u1aa0-\u1aa6\u1aa8-\u1aad\u1b4e-\u1b4f\u1b5a-\u1b60\u1b7d-\u1b7f\u1bfc-\u1bff\u1c3b-\u1c3f\u1c7e-\u1c7f\u1cc0-\u1cc7\u1cd3\u2010-\u2027\u2030-\u2043\u2045-\u2051\u2053-\u205e\u207d-\u207e\u208d-\u208e\u2308-\u230b\u2329-\u232a\u2768-\u2775\u27c5-\u27c6\u27e6-\u27ef\u2983-\u2998\u29d8-\u29db\u29fc-\u29fd\u2cf9-\u2cfc\u2cfe-\u2cff\u2d70\u2e00-\u2e2e\u2e30-\u2e4f\u2e52-\u2e5d\u3001-\u3003\u3008-\u3011\u3014-\u301f\u3030\u303d\u30a0\u30fb\ua4fe-\ua4ff\ua60d-\ua60f\ua673\ua67e\ua6f2-\ua6f7\ua874-\ua877\ua8ce-\ua8cf\ua8f8-\ua8fa\ua8fc\ua92e-\ua92f\ua95f\ua9c1-\ua9cd\ua9de-\ua9df\uaa5c-\uaa5f\uaade-\uaadf\uaaf0-\uaaf1\uabeb\ufd3e-\ufd3f\ufe10-\ufe19\ufe30-\ufe52\ufe54-\ufe61\ufe63\ufe68\ufe6a-\ufe6b\uff01-\uff03\uff05-\uff0a\uff0c-\uff0f\uff1a-\uff1b\uff1f-\uff20\uff3b-\uff3d\uff3f\uff5b\uff5d\uff5f-\uff65\U00010100-\U00010102\U0001039f\U000103d0\U0001056f\U00010857\U0001091f\U0001093f\U00010a50-\U00010a58\U00010a7f\U00010af0-\U00010af6\U00010b39-\U00010b3f\U00010b99-\U00010b9c\U00010d6e\U00010ead\U00010f55-\U00010f59\U00010f86-\U00010f89\U00011047-\U0001104d\U000110bb-\U000110bc\U000110be-\U000110c1\U00011140-\U00011143\U00011174-\U00011175\U000111c5-\U000111c8\U000111cd\U000111db\U000111dd-\U000111df\U00011238-\U0001123d\U000112a9\U000113d4-\U000113d5\U000113d7-\U000113d8\U0001144b-\U0001144f\U0001145a-\U0001145b\U0001145d\U000114c6\U000115c1-\U000115d7\U00011641-\U00011643\U00011660-\U0001166c\U000116b9\U0001173c-\U0001173e\U0001183b\U00011944-\U00011946\U000119e2\U00011a3f-\U00011a46\U00011a9a-\U00011a9c\U00011a9e-\U00011aa2\U00011b00-\U00011b09\U00011be1\U00011c41-\U00011c45\U00011c70-\U00011c71\U00011ef7-\U00011ef8\U00011f43-\U00011f4f\U00011fff\U00012470-\U00012474\U00012ff1-\U00012ff2\U00016a6e-\U00016a6f\U00016af5\U00016b37-\U00016b3b\U00016b44\U00016d6d-\U00016d6f\U00016e97-\U00016e9a\U00016fe2\U0001bc9f\U0001da87-\U0001da8b\U0001e5ff\U0001e95e-\U0001e95f]" # noqa: E501 # General_Category=Z SRC_Z = "[\\ \xa0\u1680\u2000-\u200a\u2028-\u2029\u202f\u205f\u3000]" # noqa: E501 ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987211.2111893 linkify_it_py-2.2.0/linkify_it/main.py0000644000175100017510000005622315244502413017501 0ustar00runnerrunnerimport copy import re import types from .ucre import build_re # py>=37: re.Pattern, else: _sre.SRE_Pattern RE_TYPE = type(re.compile(r"")) def _escape_re(string): return re.sub(r"([.?*+^$[\]\\(){}|-])", r"\\\1", string) def _index_of(text, search_value): try: result = text.index(search_value) except ValueError: result = -1 return result def _at(candidates, position): """Item at ``position``, or ``None`` once the list is exhausted.""" return candidates[position] if position < len(candidates) else None def _choose(a, b): """Return whichever candidate should be emitted first. The earlier start wins; on equal starts the longer match wins. A full tie keeps ``a``, so the order the candidates are folded in sets the priority. ``None`` means the corresponding list has no candidate left. """ if a is None: return b if b is None: return a if a.index != b.index: return a if a.index < b.index else b return a if a.last_index >= b.last_index else b class SchemaError(Exception): """Linkify schema error""" def __init__(self, name, val): message = f"(LinkifyIt) Invalid schema '{name}': '{val}'" super().__init__(message) class Match: """Match result. Attributes: schema (str): Prefix (protocol) for matched string. index (int): First position of matched string. last_index (int): Next position after matched string. raw (str): Matched string. text (str): Notmalized text of matched string. url (str): Normalized url of matched string. Args: text (str): text the match was found in schema (str): link schema, empty for fuzzy links index (int): first position of matched string last_index (int): next position after matched string """ def __repr__(self): return ( f"{self.__class__.__module__}.{self.__class__.__name__}({self.__dict__!r})" ) def __init__(self, text, schema, index, last_index): raw = text[index:last_index] self.schema = schema.lower() self.index = index self.last_index = last_index self.raw = raw self.text = raw self.url = raw class _Candidate: """A single hit found by one of the scan passes of :meth:`LinkifyIt.match`. Every pass collects all of its hits up front, so no pass ever rescans a tail that an earlier one already walked. Args: schema (str): link schema, empty for fuzzy links index (int): first position of matched string last_index (int): next position after matched string """ __slots__ = ("index", "last_index", "schema") def __init__(self, schema, index, last_index): self.schema = schema self.index = index self.last_index = last_index class LinkifyIt: """Creates new linkifier instance with optional additional schemas. By default understands: - ``http(s)://...`` , ``ftp://...``, ``mailto:...`` & ``//...`` links - "fuzzy" links and emails (example.com, foo@bar.com). ``schemas`` is an dict where each key/value describes protocol/rule: - **key** - link prefix (usually, protocol name with ``:`` at the end, ``skype:`` for example). `linkify-it` makes shure that prefix is not preceeded with alphanumeric char. Only whitespaces and punctuation allowed. - **value** - rule to check tail after link prefix - *str* - just alias to existing rule - *dict* - *validate* - either a ``re.Pattern``, ``re str`` (start with ``^``, and don't include the link prefix itself), or a validator ``function`` which, given arguments *self*, *text* and *pos* returns the length of a match in *text* starting at index *pos*. *pos* is the index right after the link prefix. - *normalize* - optional function to normalize text & url of matched result (for example, for @twitter mentions). ``options`` is an dict: - **fuzzyLink** - recognige URL-s without ``http(s):`` prefix. Default ``True``. - **fuzzyIP** - allow IPs in fuzzy links above. Can conflict with some texts like version numbers. Default ``False``. - **fuzzyEmail** - recognize emails without ``mailto:`` prefix. - **---** - set `True` to terminate link with `---` (if it's considered as long dash). Args: schemas (dict): Optional. Additional schemas to validate (prefix/validator) options (dict): { fuzzy_link | fuzzy_email | fuzzy_ip: True | False }. Default: {"fuzzy_link": True, "fuzzy_email": True, "fuzzy_ip": False}. """ # The built-in validators match at `pos` with a compiled pattern rather than # running an `^`-anchored search over `text[pos:]` the way upstream does. # The slice is an O(n) copy, and Python 3.10 does not apply the `^` anchor # optimization, so `re.search` rescans the whole tail -- together that is # quadratic on inputs like `mailto:mailto:...`. Schemas registered through # `add()` keep the sliced form: their patterns are documented to start # with `^`, which cannot match at a non-zero position. def _validate_http(self, text, pos): if not self.re.get("http"): # compile lazily, because "host"-containing variables can change on # tlds update. self.re["http"] = re.compile( "\\/\\/" + self.re["src_auth"] + self.re["src_host_port_strict"] + self.re["src_path"], flags=re.IGNORECASE, ) founds = self.re["http"].match(text, pos) if founds: return len(founds.group()) return 0 def _validate_double_slash(self, text, pos): if not self.re.get("not_http"): # compile lazily, because "host"-containing variables can change on # tlds update. self.re["not_http"] = re.compile( self.re["src_auth"] + "(?:localhost|(?:(?:" + self.re["src_domain"] + ")\\.)+" + self.re["src_domain_root"] + ")" + self.re["src_port"] + self.re["src_host_terminator"] + self.re["src_path"], flags=re.IGNORECASE, ) founds = self.re["not_http"].match(text, pos) if founds: if pos >= 3 and text[pos - 3] == ":": return 0 if pos >= 3 and text[pos - 3] == "/": return 0 return len(founds.group(0)) return 0 def _validate_mailto(self, text, pos): if not self.re.get("mailto"): self.re["mailto"] = re.compile( self.re["src_email_name"] + "@" + self.re["src_host_strict"], flags=re.IGNORECASE, ) founds = self.re["mailto"].match(text, pos) if founds: return len(founds.group(0)) return 0 def _create_validator(self, regex): def func(text, pos): tail = text[pos:] if isinstance(regex, str): founds = re.search(regex, tail, flags=re.IGNORECASE) else: # re.Pattern founds = re.search(regex, tail) if founds: return len(founds.group(0)) return 0 return func def _create_normalizer(self): def func(match): self.normalize(match) return func def _create_match(self, text, schema, index, last_index): match = Match(text, schema, index, last_index) self._compiled[match.schema]["normalize"](match) return match def __init__(self, schemas=None, options=None): self.default_options = { "fuzzy_link": True, "fuzzy_email": True, "fuzzy_ip": False, } self.default_schemas = { "http:": {"validate": self._validate_http}, "https:": "http:", "ftp:": "http:", "//": {"validate": self._validate_double_slash}, "mailto:": {"validate": self._validate_mailto}, } # RE pattern for 2-character tlds (autogenerated by ./support/tlds_2char_gen.js) self.tlds_2ch_src_re = "a[cdefgilmnoqrstuwxz]|b[abdefghijmnorstvwyz]|c[acdfghiklmnoruvwxyz]|d[ejkmoz]|e[cegrstu]|f[ijkmor]|g[abdefghilmnpqrstuwy]|h[kmnrtu]|i[delmnoqrst]|j[emop]|k[eghimnprwyz]|l[abcikrstuvy]|m[acdeghklmnopqrstuvwxyz]|n[acefgilopruz]|om|p[aefghklmnrstwy]|qa|r[eosuw]|s[abcdeghijklmnortuvxyz]|t[cdfghjklmnortvwz]|u[agksyz]|v[aceginu]|w[fs]|y[et]|z[amw]" # noqa: E501 # DON'T try to make PRs with changes. Extend TLDs with LinkifyIt.tlds() instead self.tlds_default = "biz|com|edu|gov|net|org|pro|web|xxx|aero|asia|coop|info|museum|name|shop|рф".split( # noqa: E501 "|" ) if options: self.default_options.update(options) self._opts = self.default_options else: self._opts = self.default_options if schemas: self.default_schemas.update(schemas) self._schemas = self.default_schemas else: self._schemas = self.default_schemas self._compiled = {} self._tlds = self.tlds_default self._tlds_replaced = False self.re = {} self._compile() def _compile(self): """Schemas compiler. Build regexps.""" # Load & clone RE patterns. self.re = build_re(self._opts) # Define dynamic patterns tlds = copy.deepcopy(self._tlds) self._on_compile() if not self._tlds_replaced: tlds.append(self.tlds_2ch_src_re) tlds.append(self.re["src_xn"]) self.re["src_tlds"] = "|".join(tlds) def untpl(tpl): return tpl.replace("%TLDS%", self.re["src_tlds"]) self.re["email_fuzzy"] = untpl(self.re["tpl_email_fuzzy"]) self.re["link_fuzzy"] = untpl(self.re["tpl_link_fuzzy"]) self.re["link_no_ip_fuzzy"] = untpl(self.re["tpl_link_no_ip_fuzzy"]) self.re["host_fuzzy_test"] = untpl(self.re["tpl_host_fuzzy_test"]) # # Compile each schema # aliases = [] self._compiled = {} for name, val in self._schemas.items(): # skip disabled methods if val is None: continue compiled = {"validate": None, "link": None} self._compiled[name] = compiled if isinstance(val, dict): if isinstance(val.get("validate"), RE_TYPE): compiled["validate"] = self._create_validator(val.get("validate")) elif isinstance(val.get("validate"), str): compiled["validate"] = self._create_validator(val.get("validate")) elif isinstance(val.get("validate"), types.MethodType): compiled["validate"] = val.get("validate") # Add custom handler elif isinstance(val.get("validate"), types.FunctionType): setattr(LinkifyIt, "func", val.get("validate")) compiled["validate"] = self.func else: raise SchemaError(name, val) if isinstance(val.get("normalize"), types.MethodType): compiled["normalize"] = val.get("normalize") # Add custom handler elif isinstance(val.get("normalize"), types.FunctionType): setattr(LinkifyIt, "func", val.get("normalize")) compiled["normalize"] = self.func elif not val.get("normalize"): compiled["normalize"] = self._create_normalizer() else: raise SchemaError(name, val) continue if isinstance(val, str): aliases.append(name) continue raise SchemaError(name, val) # # Compile postponed aliases # for alias in aliases: if not self._compiled.get(self._schemas.get(alias)): continue self._compiled[alias]["validate"] = self._compiled[self._schemas[alias]][ "validate" ] self._compiled[alias]["normalize"] = self._compiled[self._schemas[alias]][ "normalize" ] # Fake record for guessed links self._compiled[""] = {"validate": None, "normalize": self._create_normalizer()} # # Build schema condition # slist = "|".join( [ _escape_re(name) for name, val in self._compiled.items() if len(name) > 0 and val ] ) re_schema_test = ( "(^|(?!_)(?:[><\uff5c]|" + self.re["src_ZPCc"] + "))(" + slist + ")" ) # (?!_) cause 1.5x slowdown self.re["schema_test"] = re_schema_test self.re["schema_search"] = re_schema_test self.re["schema_at_start"] = "^" + self.re["schema_search"] self.re["pretest"] = ( "(" + re_schema_test + ")|(" + self.re["host_fuzzy_test"] + ")|@" ) def add(self, schema, definition): """Add new rule definition. (chainable) See :class:`linkify_it.main.LinkifyIt` init description for details. ``schema`` is a link prefix (``skype:``, for example), and ``definition`` is a ``str`` to alias to another schema, or an ``dict`` with ``validate`` and optionally `normalize` definitions. To disable an existing rule, use ``.add(, None)``. Args: schema (str): rule name (fixed pattern prefix) definition (`str` or `re.Pattern`): schema definition Return: :class:`linkify_it.main.LinkifyIt` """ self._schemas[schema] = definition self._compile() return self def set(self, options): """Override default options. (chainable) Missed properties will not be changed. Args: options (dict): ``keys``: [``fuzzy_link`` | ``fuzzy_email`` | ``fuzzy_ip``]. ``values``: [``True`` | ``False``] Return: :class:`linkify_it.main.LinkifyIt` """ self._opts.update(options) return self def test(self, text): """Searches linkifiable pattern and returns ``True`` on success or ``False`` on fail. Args: text (str): text to search Returns: bool: ``True`` if a linkable pattern was found, otherwise it is ``False``. """ if not len(text): return False if re.search(self.re["schema_test"], text, flags=re.IGNORECASE): matched_iter = re.finditer( self.re["schema_search"], text, flags=re.IGNORECASE ) for matched in matched_iter: if self.test_schema_at(text, matched.group(2), matched.end(0)): return True if self._opts.get("fuzzy_link") and self._compiled.get("http:"): # guess schemaless links if re.search(self.re["host_fuzzy_test"], text, flags=re.IGNORECASE): if self._opts.get("fuzzy_ip"): pattern = self.re["link_fuzzy"] else: pattern = self.re["link_no_ip_fuzzy"] if re.search(pattern, text, flags=re.IGNORECASE): return True if self._opts.get("fuzzy_email") and self._compiled.get("mailto:"): # guess schemaless emails at_pos = _index_of(text, "@") if at_pos >= 0: # We can't skip this check, because this cases are possible: # 192.168.1.1@gmail.com, my.in@example.com if re.search(self.re["email_fuzzy"], text, flags=re.IGNORECASE): return True return False def pretest(self, text): """Very quick check, that can give false positives. Returns true if link MAY BE can exists. Can be used for speed optimization, when you need to check that link NOT exists. Args: text (str): text to search Returns: bool: ``True`` if a linkable pattern was found, otherwise it is ``False``. """ if re.search(self.re["pretest"], text, flags=re.IGNORECASE): return True return False def test_schema_at(self, text, name, position): """Similar to :meth:`linkify_it.main.LinkifyIt.test` but checks only specific protocol tail exactly at given position. Args: text (str): text to scan name (str): rule (schema) name position (int): length of found pattern (0 on fail). Returns: int: text (str): text to search """ # If not supported schema check requested - terminate if not self._compiled.get(name.lower()): return 0 return self._compiled.get(name.lower()).get("validate")(text, position) def match(self, text): """Returns ``list`` of found link descriptions or ``None`` on fail. We strongly recommend to use :meth:`linkify_it.main.LinkifyIt.test` first, for best speed. Args: text (str): text to search Returns: ``list`` or ``None``: Result match description: * **schema** - link schema, can be empty for fuzzy links, or ``//`` for protocol-neutral links. * **index** - offset of matched text * **last_index** - offset of matched text * **raw** - offset of matched text * **text** - normalized text * **url** - link, generated from matched text """ if not len(text): return None # Collect every hit of each pattern in one pass over the whole text. schemed = [] fuzzy_link = [] fuzzy_email = [] # scan for links with schema if re.search(self.re["schema_test"], text, flags=re.IGNORECASE): matched_iter = re.finditer( self.re["schema_search"], text, flags=re.IGNORECASE ) for matched in matched_iter: length = self.test_schema_at(text, matched.group(2), matched.end(0)) if length: schemed.append( _Candidate( matched.group(2), matched.start(0) + len(matched.group(1)), matched.start(0) + len(matched.group(0)) + length, ) ) if self._opts.get("fuzzy_link") and self._compiled.get("http:"): # guess schemaless links if self._opts.get("fuzzy_ip"): pattern = self.re["link_fuzzy"] else: pattern = self.re["link_no_ip_fuzzy"] for matched in re.finditer(pattern, text, flags=re.IGNORECASE): fuzzy_link.append( _Candidate( "", matched.start(0) + len(matched.group(1)), matched.start(0) + len(matched.group(0)), ) ) if self._opts.get("fuzzy_email") and self._compiled.get("mailto:"): # guess schemaless emails matched_iter = re.finditer( self.re["email_fuzzy"], text, flags=re.IGNORECASE ) for matched in matched_iter: fuzzy_email.append( _Candidate( "mailto:", matched.start(0) + len(matched.group(1)), matched.start(0) + len(matched.group(0)), ) ) # Merge the three streams, which are each already sorted by position, # dropping candidates that overlap a match already emitted. indexes = [0, 0, 0] result = [] last_index = 0 while True: candidates = [ _at(schemed, indexes[0]), _at(fuzzy_email, indexes[1]), _at(fuzzy_link, indexes[2]), ] candidate = _choose(_choose(candidates[0], candidates[1]), candidates[2]) if candidate is None: break if candidate is candidates[0]: indexes[0] += 1 elif candidate is candidates[1]: indexes[1] += 1 else: indexes[2] += 1 if candidate.index < last_index: continue result.append( self._create_match( text, candidate.schema, candidate.index, candidate.last_index ) ) last_index = candidate.last_index if len(result): return result return None def match_at_start(self, text): """Returns fully-formed (not fuzzy) link if it starts at the beginning of the string, and null otherwise. Args: text (str): text to search Retuns: ``Match`` or ``None`` """ if not len(text): return None founds = re.search(self.re["schema_at_start"], text, flags=re.IGNORECASE) if not founds: return None length = self.test_schema_at(text, founds.group(2), len(founds.group(0))) if not length: return None return self._create_match( text, founds.group(2), founds.start(0) + len(founds.group(1)), founds.start(0) + len(founds.group(0)) + length, ) def tlds(self, list_tlds, keep_old=False): """Load (or merge) new tlds list. (chainable) Those are user for fuzzy links (without prefix) to avoid false positives. By default this algorythm used: * hostname with any 2-letter root zones are ok. * biz|com|edu|gov|net|org|pro|web|xxx|aero|asia|coop|info|museum|name|shop|рф are ok. * encoded (`xn--...`) root zones are ok. If list is replaced, then exact match for 2-chars root zones will be checked. Args: list_tlds (list or str): ``list of tlds`` or ``tlds string`` keep_old (bool): merge with current list if q`True`q (q`Falseq` by default) """ _list = list_tlds if isinstance(list_tlds, list) else [list_tlds] if not keep_old: self._tlds = _list self._tlds_replaced = True self._compile() return self self._tlds.extend(_list) self._tlds = sorted(list(set(self._tlds)), reverse=True) self._compile() return self def normalize(self, match): """Default normalizer (if schema does not define it's own). Args: match (:class:`linkify_it.main.Match`): Match result """ if not match.schema: match.url = "http://" + match.url if match.schema == "mailto:" and not re.search( "^mailto:", match.url, flags=re.IGNORECASE ): match.url = "mailto:" + match.url def _on_compile(self): """Override to modify basic RegExp-s.""" pass ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987211.2111893 linkify_it_py-2.2.0/linkify_it/tlds.py0000644000175100017510000005047015244502413017521 0ustar00runnerrunner"""TLDS Version 2020110600, Last Updated Fri Nov 6 07:07:02 2020 UTC References: http://data.iana.org/TLD/tlds-alpha-by-domain.txt """ TLDS = [ "AAA", "AARP", "ABARTH", "ABB", "ABBOTT", "ABBVIE", "ABC", "ABLE", "ABOGADO", "ABUDHABI", "AC", "ACADEMY", "ACCENTURE", "ACCOUNTANT", "ACCOUNTANTS", "ACO", "ACTOR", "AD", "ADAC", "ADS", "ADULT", "AE", "AEG", "AERO", "AETNA", "AF", "AFAMILYCOMPANY", "AFL", "AFRICA", "AG", "AGAKHAN", "AGENCY", "AI", "AIG", "AIRBUS", "AIRFORCE", "AIRTEL", "AKDN", "AL", "ALFAROMEO", "ALIBABA", "ALIPAY", "ALLFINANZ", "ALLSTATE", "ALLY", "ALSACE", "ALSTOM", "AM", "AMAZON", "AMERICANEXPRESS", "AMERICANFAMILY", "AMEX", "AMFAM", "AMICA", "AMSTERDAM", "ANALYTICS", "ANDROID", "ANQUAN", "ANZ", "AO", "AOL", "APARTMENTS", "APP", "APPLE", "AQ", "AQUARELLE", "AR", "ARAB", "ARAMCO", "ARCHI", "ARMY", "ARPA", "ART", "ARTE", "AS", "ASDA", "ASIA", "ASSOCIATES", "AT", "ATHLETA", "ATTORNEY", "AU", "AUCTION", "AUDI", "AUDIBLE", "AUDIO", "AUSPOST", "AUTHOR", "AUTO", "AUTOS", "AVIANCA", "AW", "AWS", "AX", "AXA", "AZ", "AZURE", "BA", "BABY", "BAIDU", "BANAMEX", "BANANAREPUBLIC", "BAND", "BANK", "BAR", "BARCELONA", "BARCLAYCARD", "BARCLAYS", "BAREFOOT", "BARGAINS", "BASEBALL", "BASKETBALL", "BAUHAUS", "BAYERN", "BB", "BBC", "BBT", "BBVA", "BCG", "BCN", "BD", "BE", "BEATS", "BEAUTY", "BEER", "BENTLEY", "BERLIN", "BEST", "BESTBUY", "BET", "BF", "BG", "BH", "BHARTI", "BI", "BIBLE", "BID", "BIKE", "BING", "BINGO", "BIO", "BIZ", "BJ", "BLACK", "BLACKFRIDAY", "BLOCKBUSTER", "BLOG", "BLOOMBERG", "BLUE", "BM", "BMS", "BMW", "BN", "BNPPARIBAS", "BO", "BOATS", "BOEHRINGER", "BOFA", "BOM", "BOND", "BOO", "BOOK", "BOOKING", "BOSCH", "BOSTIK", "BOSTON", "BOT", "BOUTIQUE", "BOX", "BR", "BRADESCO", "BRIDGESTONE", "BROADWAY", "BROKER", "BROTHER", "BRUSSELS", "BS", "BT", "BUDAPEST", "BUGATTI", "BUILD", "BUILDERS", "BUSINESS", "BUY", "BUZZ", "BV", "BW", "BY", "BZ", "BZH", "CA", "CAB", "CAFE", "CAL", "CALL", "CALVINKLEIN", "CAM", "CAMERA", "CAMP", "CANCERRESEARCH", "CANON", "CAPETOWN", "CAPITAL", "CAPITALONE", "CAR", "CARAVAN", "CARDS", "CARE", "CAREER", "CAREERS", "CARS", "CASA", "CASE", "CASEIH", "CASH", "CASINO", "CAT", "CATERING", "CATHOLIC", "CBA", "CBN", "CBRE", "CBS", "CC", "CD", "CEB", "CENTER", "CEO", "CERN", "CF", "CFA", "CFD", "CG", "CH", "CHANEL", "CHANNEL", "CHARITY", "CHASE", "CHAT", "CHEAP", "CHINTAI", "CHRISTMAS", "CHROME", "CHURCH", "CI", "CIPRIANI", "CIRCLE", "CISCO", "CITADEL", "CITI", "CITIC", "CITY", "CITYEATS", "CK", "CL", "CLAIMS", "CLEANING", "CLICK", "CLINIC", "CLINIQUE", "CLOTHING", "CLOUD", "CLUB", "CLUBMED", "CM", "CN", "CO", "COACH", "CODES", "COFFEE", "COLLEGE", "COLOGNE", "COM", "COMCAST", "COMMBANK", "COMMUNITY", "COMPANY", "COMPARE", "COMPUTER", "COMSEC", "CONDOS", "CONSTRUCTION", "CONSULTING", "CONTACT", "CONTRACTORS", "COOKING", "COOKINGCHANNEL", "COOL", "COOP", "CORSICA", "COUNTRY", "COUPON", "COUPONS", "COURSES", "CPA", "CR", "CREDIT", "CREDITCARD", "CREDITUNION", "CRICKET", "CROWN", "CRS", "CRUISE", "CRUISES", "CSC", "CU", "CUISINELLA", "CV", "CW", "CX", "CY", "CYMRU", "CYOU", "CZ", "DABUR", "DAD", "DANCE", "DATA", "DATE", "DATING", "DATSUN", "DAY", "DCLK", "DDS", "DE", "DEAL", "DEALER", "DEALS", "DEGREE", "DELIVERY", "DELL", "DELOITTE", "DELTA", "DEMOCRAT", "DENTAL", "DENTIST", "DESI", "DESIGN", "DEV", "DHL", "DIAMONDS", "DIET", "DIGITAL", "DIRECT", "DIRECTORY", "DISCOUNT", "DISCOVER", "DISH", "DIY", "DJ", "DK", "DM", "DNP", "DO", "DOCS", "DOCTOR", "DOG", "DOMAINS", "DOT", "DOWNLOAD", "DRIVE", "DTV", "DUBAI", "DUCK", "DUNLOP", "DUPONT", "DURBAN", "DVAG", "DVR", "DZ", "EARTH", "EAT", "EC", "ECO", "EDEKA", "EDU", "EDUCATION", "EE", "EG", "EMAIL", "EMERCK", "ENERGY", "ENGINEER", "ENGINEERING", "ENTERPRISES", "EPSON", "EQUIPMENT", "ER", "ERICSSON", "ERNI", "ES", "ESQ", "ESTATE", "ET", "ETISALAT", "EU", "EUROVISION", "EUS", "EVENTS", "EXCHANGE", "EXPERT", "EXPOSED", "EXPRESS", "EXTRASPACE", "FAGE", "FAIL", "FAIRWINDS", "FAITH", "FAMILY", "FAN", "FANS", "FARM", "FARMERS", "FASHION", "FAST", "FEDEX", "FEEDBACK", "FERRARI", "FERRERO", "FI", "FIAT", "FIDELITY", "FIDO", "FILM", "FINAL", "FINANCE", "FINANCIAL", "FIRE", "FIRESTONE", "FIRMDALE", "FISH", "FISHING", "FIT", "FITNESS", "FJ", "FK", "FLICKR", "FLIGHTS", "FLIR", "FLORIST", "FLOWERS", "FLY", "FM", "FO", "FOO", "FOOD", "FOODNETWORK", "FOOTBALL", "FORD", "FOREX", "FORSALE", "FORUM", "FOUNDATION", "FOX", "FR", "FREE", "FRESENIUS", "FRL", "FROGANS", "FRONTDOOR", "FRONTIER", "FTR", "FUJITSU", "FUJIXEROX", "FUN", "FUND", "FURNITURE", "FUTBOL", "FYI", "GA", "GAL", "GALLERY", "GALLO", "GALLUP", "GAME", "GAMES", "GAP", "GARDEN", "GAY", "GB", "GBIZ", "GD", "GDN", "GE", "GEA", "GENT", "GENTING", "GEORGE", "GF", "GG", "GGEE", "GH", "GI", "GIFT", "GIFTS", "GIVES", "GIVING", "GL", "GLADE", "GLASS", "GLE", "GLOBAL", "GLOBO", "GM", "GMAIL", "GMBH", "GMO", "GMX", "GN", "GODADDY", "GOLD", "GOLDPOINT", "GOLF", "GOO", "GOODYEAR", "GOOG", "GOOGLE", "GOP", "GOT", "GOV", "GP", "GQ", "GR", "GRAINGER", "GRAPHICS", "GRATIS", "GREEN", "GRIPE", "GROCERY", "GROUP", "GS", "GT", "GU", "GUARDIAN", "GUCCI", "GUGE", "GUIDE", "GUITARS", "GURU", "GW", "GY", "HAIR", "HAMBURG", "HANGOUT", "HAUS", "HBO", "HDFC", "HDFCBANK", "HEALTH", "HEALTHCARE", "HELP", "HELSINKI", "HERE", "HERMES", "HGTV", "HIPHOP", "HISAMITSU", "HITACHI", "HIV", "HK", "HKT", "HM", "HN", "HOCKEY", "HOLDINGS", "HOLIDAY", "HOMEDEPOT", "HOMEGOODS", "HOMES", "HOMESENSE", "HONDA", "HORSE", "HOSPITAL", "HOST", "HOSTING", "HOT", "HOTELES", "HOTELS", "HOTMAIL", "HOUSE", "HOW", "HR", "HSBC", "HT", "HU", "HUGHES", "HYATT", "HYUNDAI", "IBM", "ICBC", "ICE", "ICU", "ID", "IE", "IEEE", "IFM", "IKANO", "IL", "IM", "IMAMAT", "IMDB", "IMMO", "IMMOBILIEN", "IN", "INC", "INDUSTRIES", "INFINITI", "INFO", "ING", "INK", "INSTITUTE", "INSURANCE", "INSURE", "INT", "INTERNATIONAL", "INTUIT", "INVESTMENTS", "IO", "IPIRANGA", "IQ", "IR", "IRISH", "IS", "ISMAILI", "IST", "ISTANBUL", "IT", "ITAU", "ITV", "IVECO", "JAGUAR", "JAVA", "JCB", "JCP", "JE", "JEEP", "JETZT", "JEWELRY", "JIO", "JLL", "JM", "JMP", "JNJ", "JO", "JOBS", "JOBURG", "JOT", "JOY", "JP", "JPMORGAN", "JPRS", "JUEGOS", "JUNIPER", "KAUFEN", "KDDI", "KE", "KERRYHOTELS", "KERRYLOGISTICS", "KERRYPROPERTIES", "KFH", "KG", "KH", "KI", "KIA", "KIM", "KINDER", "KINDLE", "KITCHEN", "KIWI", "KM", "KN", "KOELN", "KOMATSU", "KOSHER", "KP", "KPMG", "KPN", "KR", "KRD", "KRED", "KUOKGROUP", "KW", "KY", "KYOTO", "KZ", "LA", "LACAIXA", "LAMBORGHINI", "LAMER", "LANCASTER", "LANCIA", "LAND", "LANDROVER", "LANXESS", "LASALLE", "LAT", "LATINO", "LATROBE", "LAW", "LAWYER", "LB", "LC", "LDS", "LEASE", "LECLERC", "LEFRAK", "LEGAL", "LEGO", "LEXUS", "LGBT", "LI", "LIDL", "LIFE", "LIFEINSURANCE", "LIFESTYLE", "LIGHTING", "LIKE", "LILLY", "LIMITED", "LIMO", "LINCOLN", "LINDE", "LINK", "LIPSY", "LIVE", "LIVING", "LIXIL", "LK", "LLC", "LLP", "LOAN", "LOANS", "LOCKER", "LOCUS", "LOFT", "LOL", "LONDON", "LOTTE", "LOTTO", "LOVE", "LPL", "LPLFINANCIAL", "LR", "LS", "LT", "LTD", "LTDA", "LU", "LUNDBECK", "LUPIN", "LUXE", "LUXURY", "LV", "LY", "MA", "MACYS", "MADRID", "MAIF", "MAISON", "MAKEUP", "MAN", "MANAGEMENT", "MANGO", "MAP", "MARKET", "MARKETING", "MARKETS", "MARRIOTT", "MARSHALLS", "MASERATI", "MATTEL", "MBA", "MC", "MCKINSEY", "MD", "ME", "MED", "MEDIA", "MEET", "MELBOURNE", "MEME", "MEMORIAL", "MEN", "MENU", "MERCKMSD", "MG", "MH", "MIAMI", "MICROSOFT", "MIL", "MINI", "MINT", "MIT", "MITSUBISHI", "MK", "ML", "MLB", "MLS", "MM", "MMA", "MN", "MO", "MOBI", "MOBILE", "MODA", "MOE", "MOI", "MOM", "MONASH", "MONEY", "MONSTER", "MORMON", "MORTGAGE", "MOSCOW", "MOTO", "MOTORCYCLES", "MOV", "MOVIE", "MP", "MQ", "MR", "MS", "MSD", "MT", "MTN", "MTR", "MU", "MUSEUM", "MUTUAL", "MV", "MW", "MX", "MY", "MZ", "NA", "NAB", "NAGOYA", "NAME", "NATIONWIDE", "NATURA", "NAVY", "NBA", "NC", "NE", "NEC", "NET", "NETBANK", "NETFLIX", "NETWORK", "NEUSTAR", "NEW", "NEWHOLLAND", "NEWS", "NEXT", "NEXTDIRECT", "NEXUS", "NF", "NFL", "NG", "NGO", "NHK", "NI", "NICO", "NIKE", "NIKON", "NINJA", "NISSAN", "NISSAY", "NL", "NO", "NOKIA", "NORTHWESTERNMUTUAL", "NORTON", "NOW", "NOWRUZ", "NOWTV", "NP", "NR", "NRA", "NRW", "NTT", "NU", "NYC", "NZ", "OBI", "OBSERVER", "OFF", "OFFICE", "OKINAWA", "OLAYAN", "OLAYANGROUP", "OLDNAVY", "OLLO", "OM", "OMEGA", "ONE", "ONG", "ONL", "ONLINE", "ONYOURSIDE", "OOO", "OPEN", "ORACLE", "ORANGE", "ORG", "ORGANIC", "ORIGINS", "OSAKA", "OTSUKA", "OTT", "OVH", "PA", "PAGE", "PANASONIC", "PARIS", "PARS", "PARTNERS", "PARTS", "PARTY", "PASSAGENS", "PAY", "PCCW", "PE", "PET", "PF", "PFIZER", "PG", "PH", "PHARMACY", "PHD", "PHILIPS", "PHONE", "PHOTO", "PHOTOGRAPHY", "PHOTOS", "PHYSIO", "PICS", "PICTET", "PICTURES", "PID", "PIN", "PING", "PINK", "PIONEER", "PIZZA", "PK", "PL", "PLACE", "PLAY", "PLAYSTATION", "PLUMBING", "PLUS", "PM", "PN", "PNC", "POHL", "POKER", "POLITIE", "PORN", "POST", "PR", "PRAMERICA", "PRAXI", "PRESS", "PRIME", "PRO", "PROD", "PRODUCTIONS", "PROF", "PROGRESSIVE", "PROMO", "PROPERTIES", "PROPERTY", "PROTECTION", "PRU", "PRUDENTIAL", "PS", "PT", "PUB", "PW", "PWC", "PY", "QA", "QPON", "QUEBEC", "QUEST", "QVC", "RACING", "RADIO", "RAID", "RE", "READ", "REALESTATE", "REALTOR", "REALTY", "RECIPES", "RED", "REDSTONE", "REDUMBRELLA", "REHAB", "REISE", "REISEN", "REIT", "RELIANCE", "REN", "RENT", "RENTALS", "REPAIR", "REPORT", "REPUBLICAN", "REST", "RESTAURANT", "REVIEW", "REVIEWS", "REXROTH", "RICH", "RICHARDLI", "RICOH", "RIL", "RIO", "RIP", "RMIT", "RO", "ROCHER", "ROCKS", "RODEO", "ROGERS", "ROOM", "RS", "RSVP", "RU", "RUGBY", "RUHR", "RUN", "RW", "RWE", "RYUKYU", "SA", "SAARLAND", "SAFE", "SAFETY", "SAKURA", "SALE", "SALON", "SAMSCLUB", "SAMSUNG", "SANDVIK", "SANDVIKCOROMANT", "SANOFI", "SAP", "SARL", "SAS", "SAVE", "SAXO", "SB", "SBI", "SBS", "SC", "SCA", "SCB", "SCHAEFFLER", "SCHMIDT", "SCHOLARSHIPS", "SCHOOL", "SCHULE", "SCHWARZ", "SCIENCE", "SCJOHNSON", "SCOT", "SD", "SE", "SEARCH", "SEAT", "SECURE", "SECURITY", "SEEK", "SELECT", "SENER", "SERVICES", "SES", "SEVEN", "SEW", "SEX", "SEXY", "SFR", "SG", "SH", "SHANGRILA", "SHARP", "SHAW", "SHELL", "SHIA", "SHIKSHA", "SHOES", "SHOP", "SHOPPING", "SHOUJI", "SHOW", "SHOWTIME", "SHRIRAM", "SI", "SILK", "SINA", "SINGLES", "SITE", "SJ", "SK", "SKI", "SKIN", "SKY", "SKYPE", "SL", "SLING", "SM", "SMART", "SMILE", "SN", "SNCF", "SO", "SOCCER", "SOCIAL", "SOFTBANK", "SOFTWARE", "SOHU", "SOLAR", "SOLUTIONS", "SONG", "SONY", "SOY", "SPA", "SPACE", "SPORT", "SPOT", "SPREADBETTING", "SR", "SRL", "SS", "ST", "STADA", "STAPLES", "STAR", "STATEBANK", "STATEFARM", "STC", "STCGROUP", "STOCKHOLM", "STORAGE", "STORE", "STREAM", "STUDIO", "STUDY", "STYLE", "SU", "SUCKS", "SUPPLIES", "SUPPLY", "SUPPORT", "SURF", "SURGERY", "SUZUKI", "SV", "SWATCH", "SWIFTCOVER", "SWISS", "SX", "SY", "SYDNEY", "SYSTEMS", "SZ", "TAB", "TAIPEI", "TALK", "TAOBAO", "TARGET", "TATAMOTORS", "TATAR", "TATTOO", "TAX", "TAXI", "TC", "TCI", "TD", "TDK", "TEAM", "TECH", "TECHNOLOGY", "TEL", "TEMASEK", "TENNIS", "TEVA", "TF", "TG", "TH", "THD", "THEATER", "THEATRE", "TIAA", "TICKETS", "TIENDA", "TIFFANY", "TIPS", "TIRES", "TIROL", "TJ", "TJMAXX", "TJX", "TK", "TKMAXX", "TL", "TM", "TMALL", "TN", "TO", "TODAY", "TOKYO", "TOOLS", "TOP", "TORAY", "TOSHIBA", "TOTAL", "TOURS", "TOWN", "TOYOTA", "TOYS", "TR", "TRADE", "TRADING", "TRAINING", "TRAVEL", "TRAVELCHANNEL", "TRAVELERS", "TRAVELERSINSURANCE", "TRUST", "TRV", "TT", "TUBE", "TUI", "TUNES", "TUSHU", "TV", "TVS", "TW", "TZ", "UA", "UBANK", "UBS", "UG", "UK", "UNICOM", "UNIVERSITY", "UNO", "UOL", "UPS", "US", "UY", "UZ", "VA", "VACATIONS", "VANA", "VANGUARD", "VC", "VE", "VEGAS", "VENTURES", "VERISIGN", "VERSICHERUNG", "VET", "VG", "VI", "VIAJES", "VIDEO", "VIG", "VIKING", "VILLAS", "VIN", "VIP", "VIRGIN", "VISA", "VISION", "VIVA", "VIVO", "VLAANDEREN", "VN", "VODKA", "VOLKSWAGEN", "VOLVO", "VOTE", "VOTING", "VOTO", "VOYAGE", "VU", "VUELOS", "WALES", "WALMART", "WALTER", "WANG", "WANGGOU", "WATCH", "WATCHES", "WEATHER", "WEATHERCHANNEL", "WEBCAM", "WEBER", "WEBSITE", "WED", "WEDDING", "WEIBO", "WEIR", "WF", "WHOSWHO", "WIEN", "WIKI", "WILLIAMHILL", "WIN", "WINDOWS", "WINE", "WINNERS", "WME", "WOLTERSKLUWER", "WOODSIDE", "WORK", "WORKS", "WORLD", "WOW", "WS", "WTC", "WTF", "XBOX", "XEROX", "XFINITY", "XIHUAN", "XIN", "XN--11B4C3D", "XN--1CK2E1B", "XN--1QQW23A", "XN--2SCRJ9C", "XN--30RR7Y", "XN--3BST00M", "XN--3DS443G", "XN--3E0B707E", "XN--3HCRJ9C", "XN--3OQ18VL8PN36A", "XN--3PXU8K", "XN--42C2D9A", "XN--45BR5CYL", "XN--45BRJ9C", "XN--45Q11C", "XN--4GBRIM", "XN--54B7FTA0CC", "XN--55QW42G", "XN--55QX5D", "XN--5SU34J936BGSG", "XN--5TZM5G", "XN--6FRZ82G", "XN--6QQ986B3XL", "XN--80ADXHKS", "XN--80AO21A", "XN--80AQECDR1A", "XN--80ASEHDB", "XN--80ASWG", "XN--8Y0A063A", "XN--90A3AC", "XN--90AE", "XN--90AIS", "XN--9DBQ2A", "XN--9ET52U", "XN--9KRT00A", "XN--B4W605FERD", "XN--BCK1B9A5DRE4C", "XN--C1AVG", "XN--C2BR7G", "XN--CCK2B3B", "XN--CCKWCXETD", "XN--CG4BKI", "XN--CLCHC0EA0B2G2A9GCD", "XN--CZR694B", "XN--CZRS0T", "XN--CZRU2D", "XN--D1ACJ3B", "XN--D1ALF", "XN--E1A4C", "XN--ECKVDTC9D", "XN--EFVY88H", "XN--FCT429K", "XN--FHBEI", "XN--FIQ228C5HS", "XN--FIQ64B", "XN--FIQS8S", "XN--FIQZ9S", "XN--FJQ720A", "XN--FLW351E", "XN--FPCRJ9C3D", "XN--FZC2C9E2C", "XN--FZYS8D69UVGM", "XN--G2XX48C", "XN--GCKR3F0F", "XN--GECRJ9C", "XN--GK3AT1E", "XN--H2BREG3EVE", "XN--H2BRJ9C", "XN--H2BRJ9C8C", "XN--HXT814E", "XN--I1B6B1A6A2E", "XN--IMR513N", "XN--IO0A7I", "XN--J1AEF", "XN--J1AMH", "XN--J6W193G", "XN--JLQ480N2RG", "XN--JLQ61U9W7B", "XN--JVR189M", "XN--KCRX77D1X4A", "XN--KPRW13D", "XN--KPRY57D", "XN--KPUT3I", "XN--L1ACC", "XN--LGBBAT1AD8J", "XN--MGB9AWBF", "XN--MGBA3A3EJT", "XN--MGBA3A4F16A", "XN--MGBA7C0BBN0A", "XN--MGBAAKC7DVF", "XN--MGBAAM7A8H", "XN--MGBAB2BD", "XN--MGBAH1A3HJKRD", "XN--MGBAI9AZGQP6J", "XN--MGBAYH7GPA", "XN--MGBBH1A", "XN--MGBBH1A71E", "XN--MGBC0A9AZCG", "XN--MGBCA7DZDO", "XN--MGBCPQ6GPA1A", "XN--MGBERP4A5D4AR", "XN--MGBGU82A", "XN--MGBI4ECEXP", "XN--MGBPL2FH", "XN--MGBT3DHD", "XN--MGBTX2B", "XN--MGBX4CD0AB", "XN--MIX891F", "XN--MK1BU44C", "XN--MXTQ1M", "XN--NGBC5AZD", "XN--NGBE9E0A", "XN--NGBRX", "XN--NODE", "XN--NQV7F", "XN--NQV7FS00EMA", "XN--NYQY26A", "XN--O3CW4H", "XN--OGBPF8FL", "XN--OTU796D", "XN--P1ACF", "XN--P1AI", "XN--PGBS0DH", "XN--PSSY2U", "XN--Q7CE6A", "XN--Q9JYB4C", "XN--QCKA1PMC", "XN--QXA6A", "XN--QXAM", "XN--RHQV96G", "XN--ROVU88B", "XN--RVC1E0AM3E", "XN--S9BRJ9C", "XN--SES554G", "XN--T60B56A", "XN--TCKWE", "XN--TIQ49XQYJ", "XN--UNUP4Y", "XN--VERMGENSBERATER-CTB", "XN--VERMGENSBERATUNG-PWB", "XN--VHQUV", "XN--VUQ861B", "XN--W4R85EL8FHU5DNRA", "XN--W4RS40L", "XN--WGBH1C", "XN--WGBL6A", "XN--XHQ521B", "XN--XKC2AL3HYE2A", "XN--XKC2DL3A5EE0H", "XN--Y9A3AQ", "XN--YFRO4I67O", "XN--YGBI2AMMX", "XN--ZFR164B", "XXX", "XYZ", "YACHTS", "YAHOO", "YAMAXUN", "YANDEX", "YE", "YODOBASHI", "YOGA", "YOKOHAMA", "YOU", "YOUTUBE", "YT", "YUN", "ZA", "ZAPPOS", "ZARA", "ZERO", "ZIP", "ZM", "ZONE", "ZUERICH", "ZW", ] ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987211.2111893 linkify_it_py-2.2.0/linkify_it/ucre.py0000644000175100017510000001615115244502413017507 0ustar00runnerrunnerfrom ._ucre_data import SRC_ANY, SRC_CC, SRC_CF, SRC_P, SRC_Z # \p{\Z\P\Cc\CF} (white spaces + control + format + punctuation) SRC_ZPCC = "|".join([SRC_Z, SRC_P, SRC_CC]) # \p{\Z\Cc} (white spaces + control) SRC_ZCC = "|".join([SRC_Z, SRC_CC]) # Experimental. List of chars, completely prohibited in links # because can separate it from other part of text TEXT_SEPARATORS = "[><\uff5c]" # All possible word characters (everything without punctuation, spaces & controls) # Defined via punctuation & spaces to save space # Should be something like \p{\L\N\S\M} (\w but without `_`) SRC_PSEUDO_LETTER = "(?:(?!" + TEXT_SEPARATORS + "|" + SRC_ZPCC + ")" + SRC_ANY + ")" # The same as abothe but without [0-9] # var SRC_PSEUDO_LETTER_non_d = '(?:(?![0-9]|' + SRC_ZPCC + ')' + SRC_ANY + ')' # ============================================================================= SRC_IP4 = ( "(?:(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(25[0-5]|" + "2[0-4][0-9]|[01]?[0-9][0-9]?)" ) # Prohibit any of "@/[]()" in user/pass to avoid wrong domain fetch. # Length is capped to exclude possible rescans till the end and avoid O(n^2) # DoS. No standard limit, just take something reasonable. SRC_AUTH = "(?:(?:(?!" + SRC_ZCC + "|[@/\\[\\]()]).){1,50}@)?" SRC_PORT = ( "(?::(?:6(?:[0-4]\\d{3}|5(?:[0-4]\\d{2}|5(?:[0-2]\\d|3[0-5])))|[1-5]?\\d{1,4}))?" ) # Allow anything in markdown spec, forbid quote (") at the first position # because emails enclosed in quotes are far more common # Max name length capped to 64 chars (RFC 5321). This also prevents O(n^2) # rescans to the end on inputs like `mailto:mailto:...` SRC_EMAIL_NAME = '[\\-;:&=\\+\\$,\\.a-zA-Z0-9_][\\-;:&=\\+\\$,\\"\\.a-zA-Z0-9_]{0,63}' SRC_XN = "xn--[a-z0-9\\-]{1,59}" # More to read about domain names # http:#serverfault.com/questions/638260/ # Allow letters & digits (http:#test1) SRC_DOMAIN_ROOT = "(?:" + SRC_XN + "|" + SRC_PSEUDO_LETTER + "{1,63}" + ")" SRC_DOMAIN = ( "(?:" + SRC_XN + "|" + "(?:" + SRC_PSEUDO_LETTER + ")" + "|" + "(?:" + SRC_PSEUDO_LETTER + "(?:-|" + SRC_PSEUDO_LETTER + "){0,61}" + SRC_PSEUDO_LETTER + ")" + ")" ) SRC_HOST = ( "(?:" + # Don't need IP check, because digits are already allowed in normal domain names # SRC_IP4 + # '|' + "(?:(?:(?:" + SRC_DOMAIN + ")\\.)*" + SRC_DOMAIN # _root + ")" + ")" ) TPL_HOST_FUZZY = ( "(?:" + SRC_IP4 + "|" + "(?:(?:(?:" + SRC_DOMAIN + ")\\.)+(?:%TLDS%))" + ")" ) TPL_HOST_NO_IP_FUZZY = "(?:(?:(?:" + SRC_DOMAIN + ")\\.)+(?:%TLDS%))" # ============================================================================= # Rude test fuzzy links by host, for quick deny TPL_HOST_FUZZY_TEST = ( "localhost|www\\.|\\.\\d{1,3}\\.|(?:\\.(?:%TLDS%)(?:" + SRC_ZPCC + "|>|$))" ) def _re_host_terminator(opts): src_host_terminator = ( "(?=$|" + TEXT_SEPARATORS + "|" + SRC_ZPCC + ")" + "(?!" + ("-(?!--)|" if opts.get("---") else "-|") + "_|:\\d|\\.-|\\.(?!$|" + SRC_ZPCC + "))" ) return src_host_terminator def _re_src_path(opts): src_path = ( "(?:" + "[/?#]" + "(?:" + "(?!" + SRC_ZCC + "|" + TEXT_SEPARATORS + "|[()[\\]{}.,\"'?!\\-;]).|" + "\\[(?:(?!" + SRC_ZCC + "|\\]).)*\\]|" + "\\((?:(?!" + SRC_ZCC + "|[)]).)*\\)|" + "\\{(?:(?!" + SRC_ZCC + "|[}]).)*\\}|" + '\\"(?:(?!' + SRC_ZCC + '|["]).)+\\"|' + "\\'(?:(?!" + SRC_ZCC + "|[']).)+\\'|" + "\\'(?=" + SRC_PSEUDO_LETTER + "|[-])|" + "\\.{2,}[a-zA-Z0-9%/&]|" # google has many dots in "google search" links (#66, #81). # github has ... in commit range links, # ReSTRICT to # - english # - percent-encoded # - parts of file path # - params separator # until more examples found. + "\\.(?!" + SRC_ZCC + "|[.]|$)|" + ("\\-(?!--(?:[^-]|$))(?:-*)|" if opts.get("---") else "\\-+|") + ",(?!" + SRC_ZCC + "|$)|" # allow `,,,` in paths + ";(?!" + SRC_ZCC + "|$)|" # allow `,,,` in paths + "\\!+(?!" + SRC_ZCC + "|[!]|$)|" # allow `!!!` in paths, but not at the end + "\\?(?!" + SRC_ZCC + "|[?]|$)" + ")+" + "|\\/" + ")?" ) return src_path def build_re(opts): """Build regex Args: opts (dict): options Return: dict: dict of regex string """ SRC_HOST_STRICT = SRC_HOST + _re_host_terminator(opts) TPL_HOST_FUZZY_STRICT = TPL_HOST_FUZZY + _re_host_terminator(opts) SRC_HOST_PORT_STRICT = SRC_HOST + SRC_PORT + _re_host_terminator(opts) TPL_HOST_PORT_FUZZY_STRICT = TPL_HOST_FUZZY + SRC_PORT + _re_host_terminator(opts) TPL_HOST_PORT_NO_IP_FUZZY_STRICT = ( TPL_HOST_NO_IP_FUZZY + SRC_PORT + _re_host_terminator(opts) ) TPL_EMAIL_FUZZY = ( "(^|" + TEXT_SEPARATORS + '|"|\\(|' + SRC_ZCC + ")" + "(" + SRC_EMAIL_NAME + "@" + TPL_HOST_FUZZY_STRICT + ")" ) regex = { "src_Any": SRC_ANY, "src_Cc": SRC_CC, "src_Cf": SRC_CF, "src_Z": SRC_Z, "src_P": SRC_P, "src_ZPCc": SRC_ZPCC, "src_ZCc": SRC_ZCC, "src_pseudo_letter": SRC_PSEUDO_LETTER, "src_ip4": SRC_IP4, "src_auth": SRC_AUTH, "src_port": SRC_PORT, "src_host_terminator": _re_host_terminator(opts), "src_path": _re_src_path(opts), "src_email_name": SRC_EMAIL_NAME, "src_xn": SRC_XN, "src_domain_root": SRC_DOMAIN_ROOT, "src_domain": SRC_DOMAIN, "src_host": SRC_HOST, "tpl_host_fuzzy": TPL_HOST_FUZZY, "tpl_host_no_ip_fuzzy": TPL_HOST_NO_IP_FUZZY, "src_host_strict": SRC_HOST_STRICT, "tpl_host_fuzzy_strict": TPL_HOST_FUZZY_STRICT, "src_host_port_strict": SRC_HOST_PORT_STRICT, "tpl_host_port_fuzzy_strict": TPL_HOST_PORT_FUZZY_STRICT, "tpl_host_port_no_ip_fuzzy_strict": TPL_HOST_PORT_FUZZY_STRICT, # Main rules "tpl_host_fuzzy_test": TPL_HOST_FUZZY_TEST, "tpl_email_fuzzy": TPL_EMAIL_FUZZY, # Fuzzy link can't be prepended with .:/\- and non punctuation. # but can start with > (markdown blockquote) "tpl_link_fuzzy": ( "(^|(?![.:/\\-_@])(?:[$+<=>^`|\uff5c]|" + SRC_ZPCC + "))" + "((?![$+<=>^`|\uff5c])" + TPL_HOST_PORT_FUZZY_STRICT + _re_src_path(opts) + ")" ), # Fuzzy link can't be prepended with .:/\- and non punctuation. # but can start with > (markdown blockquote) "tpl_link_no_ip_fuzzy": ( "(^|(?![.:/\\-_@])(?:[$+<=>^`|\uff5c]|" + SRC_ZPCC + "))" + "((?![$+<=>^`|\uff5c])" + TPL_HOST_PORT_NO_IP_FUZZY_STRICT + _re_src_path(opts) + ")" ), } return regex ././@PaxHeader0000000000000000000000000000003200000000000010210 xustar0026 mtime=1787987214.92316 linkify_it_py-2.2.0/linkify_it_py.egg-info/0000755000175100017510000000000015244502417020401 5ustar00runnerrunner././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987214.9075367 linkify_it_py-2.2.0/linkify_it_py.egg-info/PKG-INFO0000644000175100017510000002055115244502417021501 0ustar00runnerrunnerMetadata-Version: 2.4 Name: linkify-it-py Version: 2.2.0 Summary: Links recognition library with FULL unicode support. Author: tsutsu3 License: MIT Project-URL: Homepage, https://github.com/tsutsu3/linkify-it-py Keywords: linkify,linkifier,autolink,autolinker Classifier: Development Status :: 5 - Production/Stable Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.10 Classifier: Programming Language :: Python :: 3.11 Classifier: Programming Language :: Python :: 3.12 Classifier: Programming Language :: Python :: 3.13 Classifier: Programming Language :: Python :: 3.14 Classifier: License :: OSI Approved :: MIT License Classifier: Operating System :: OS Independent Classifier: Intended Audience :: Developers Classifier: Topic :: Software Development :: Libraries :: Python Modules Requires-Python: >=3.10 Description-Content-Type: text/markdown License-File: LICENSE Provides-Extra: test Requires-Dist: pytest; extra == "test" Requires-Dist: coverage; extra == "test" Requires-Dist: pytest-cov; extra == "test" Requires-Dist: pytest-timeout; extra == "test" Provides-Extra: dev Requires-Dist: pre-commit; extra == "dev" Requires-Dist: isort; extra == "dev" Requires-Dist: flake8; extra == "dev" Requires-Dist: black; extra == "dev" Requires-Dist: pyproject-flake8; extra == "dev" Provides-Extra: benchmark Requires-Dist: pytest; extra == "benchmark" Requires-Dist: pytest-benchmark; extra == "benchmark" Provides-Extra: doc Requires-Dist: sphinx; extra == "doc" Requires-Dist: sphinx_book_theme; extra == "doc" Requires-Dist: myst-parser; extra == "doc" Dynamic: license-file # linkify-it-py [![CI](https://github.com/tsutsu3/linkify-it-py/actions/workflows/github-ci.yml/badge.svg)](https://github.com/tsutsu3/linkify-it-py/actions/workflows/github-ci.yml) [![pypi](https://img.shields.io/pypi/v/linkify-it-py)](https://pypi.org/project/linkify-it-py/) [![Anaconda-Server Badge](https://anaconda.org/conda-forge/linkify-it-py/badges/version.svg)](https://anaconda.org/conda-forge/linkify-it-py) [![Documentation Status](https://readthedocs.org/projects/linkify-it-py/badge/?version=latest)](https://linkify-it-py.readthedocs.io/en/latest/?badge=latest) [![codecov](https://codecov.io/gh/tsutsu3/linkify-it-py/branch/main/graph/badge.svg)](https://codecov.io/gh/tsutsu3/linkify-it-py) This is Python port of [linkify-it](https://github.com/markdown-it/linkify-it). > Links recognition library with FULL unicode support. > Focused on high quality link patterns detection in plain text. __[Demo](https://linkify-it-py-demo.vercel.app/)__ __[Javascript Demo](http://markdown-it.github.io/linkify-it/)__ Why it's awesome: - Full unicode support, _with astral characters_! - International domains support. - Allows rules extension & custom normalizers. ## Python Version Support Tested on Python 3.10–3.14. The primary version for code coverage follows the latest [security phase](https://devguide.python.org/versions/) release (currently __3.12__). ## Install ```bash pip install linkify-it-py ``` or ```bash conda install -c conda-forge linkify-it-py ``` ## Usage examples ### Example 1. Simple use ```python from linkify_it import LinkifyIt linkify = LinkifyIt() print(linkify.test("Site github.com!")) # => True print(linkify.match("Site github.com!")) # => [linkify_it.main.Match({ # 'schema': '', # 'index': 5, # 'last_index': 15, # 'raw': 'github.com', # 'text': 'github.com', # 'url': 'http://github.com' # }] ``` ### Example 2. With options ```python from linkify_it import LinkifyIt from linkify_it.tlds import TLDS # Reload full tlds list & add unofficial `.onion` domain. linkify = ( LinkifyIt() .tlds(TLDS) # Reload with full tlds list .tlds("onion", True) # Add unofficial `.onion` domain .add("git:", "http:") # Add `git:` protocol as "alias" .add("ftp:", None) # Disable `ftp:` protocol .set({"fuzzy_ip": True}) # Enable IPs in fuzzy links (without schema) ) print(linkify.test("Site tamanegi.onion!")) # => True print(linkify.match("Site tamanegi.onion!")) # => [linkify_it.main.Match({ # 'schema': '', # 'index': 5, # 'last_index': 19, # 'raw': 'tamanegi.onion', # 'text': 'tamanegi.onion', # 'url': 'http://tamanegi.onion' # }] ``` ### Example 3. Add twitter mentions handler ```python from linkify_it import LinkifyIt linkify = LinkifyIt() def validate(obj, text, pos): tail = text[pos:] if not obj.re.get("twitter"): obj.re["twitter"] = re.compile( "^([a-zA-Z0-9_]){1,15}(?!_)(?=$|" + obj.re["src_ZPCc"] + ")" ) if obj.re["twitter"].search(tail): if pos > 2 and tail[pos - 2] == "@": return False return len(obj.re["twitter"].search(tail).group()) return 0 def normalize(obj, match): match.url = "https://twitter.com/" + re.sub(r"^@", "", match.url) linkify.add("@", {"validate": validate, "normalize": normalize}) ``` ## API [API documentation](https://linkify-it-py.readthedocs.io/en/latest/) ### LinkifyIt(schemas, options) Creates new linkifier instance with optional additional schemas. By default understands: - `http(s)://...` , `ftp://...`, `mailto:...` & `//...` links - "fuzzy" links and emails (google.com, foo@bar.com). `schemas` is an dict, where each key/value describes protocol/rule: - __key__ - link prefix (usually, protocol name with `:` at the end, `skype:` for example). `linkify-it-py` makes sure that prefix is not preceded with alphanumeric char. - __value__ - rule to check tail after link prefix - _str_ - just alias to existing rule - _dict_ - _validate_ - either a `re.Pattern` (start with `^`, and don't include the link prefix itself), or a validator `function` which, given arguments _self_, _text_ and _pos_, returns the length of a match in _text_ starting at index _pos_. _pos_ is the index right after the link prefix. _self_ can be used to access the linkify object to cache data. - _normalize_ - optional function to normalize text & url of matched result (for example, for twitter mentions). `options`: - __fuzzy_link__ - recognize URL-s without `http(s)://` head. Default `True`. - __fuzzy_ip__ - allow IPs in fuzzy links above. Can conflict with some texts like version numbers. Default `False`. - __fuzzy_email__ - recognize emails without `mailto:` prefix. Default `True`. - __---__ - set `True` to terminate link with `---` (if it's considered as long dash). ### .test(text) Searches linkifiable pattern and returns `True` on success or `False` on fail. ### .pretest(text) Quick check if link MAY BE can exist. Can be used to optimize more expensive `.test()` calls. Return `False` if link can not be found, `True` - if `.test()` call needed to know exactly. ### .test_schema_at(text, name, position) Similar to `.test()` but checks only specific protocol tail exactly at given position. Returns length of found pattern (0 on fail). ### .match(text) Returns `list` of found link matches or null if nothing found. Each match has: - __schema__ - link schema, can be empty for fuzzy links, or `//` for protocol-neutral links. - __index__ - offset of matched text - __last_index__ - index of next char after mathch end - __raw__ - matched text - __text__ - normalized text - __url__ - link, generated from matched text ### .matchAtStart(text) Checks if a match exists at the start of the string. Returns `Match` (see docs for `match(text)`) or null if no URL is at the start. Doesn't work with fuzzy links. ### .tlds(list_tlds, keep_old=False) Load (or merge) new tlds list. Those are needed for fuzzy links (without schema) to avoid false positives. By default: - 2-letter root zones are ok. - biz|com|edu|gov|net|org|pro|web|xxx|aero|asia|coop|info|museum|name|shop|рф are ok. - encoded (`xn--...`) root zones are ok. If that's not enough, you can reload defaults with more detailed zones list. ### .add(key, value) Add a new schema to the schemas object. As described in the constructor definition, `key` is a link prefix (`skype:`, for example), and `value` is a `str` to alias to another schema, or an `dict` with `validate` and optionally `normalize` definitions. To disable an existing rule, use `.add(key, None)`. ### .set(options) Override default options. Missed properties will not be changed. ## License [MIT](https://github.com/tsutsu3/linkify-it-py/blob/master/LICENSE) ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987214.9163582 linkify_it_py-2.2.0/linkify_it_py.egg-info/SOURCES.txt0000644000175100017510000000105415244502417022265 0ustar00runnerrunnerCHANGELOG.md LICENSE MANIFEST.in README.md pyproject.toml tox.ini linkify_it/__init__.py linkify_it/_ucre_data.py linkify_it/main.py linkify_it/tlds.py linkify_it/ucre.py linkify_it_py.egg-info/PKG-INFO linkify_it_py.egg-info/SOURCES.txt linkify_it_py.egg-info/dependency_links.txt linkify_it_py.egg-info/requires.txt linkify_it_py.egg-info/top_level.txt test/__init__.py test/test_apis.py test/test_linkify.py test/test_pathological.py test/test_port_parity.py test/utils.py test/fixtures/links.txt test/fixtures/not_links.txt test/fixtures/port_links.txt././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1787987214.908592 linkify_it_py-2.2.0/linkify_it_py.egg-info/dependency_links.txt0000644000175100017510000000000115244502417024447 0ustar00runnerrunner ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987214.9092498 linkify_it_py-2.2.0/linkify_it_py.egg-info/requires.txt0000644000175100017510000000027115244502417023001 0ustar00runnerrunner [benchmark] pytest pytest-benchmark [dev] pre-commit isort flake8 black pyproject-flake8 [doc] sphinx sphinx_book_theme myst-parser [test] pytest coverage pytest-cov pytest-timeout ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987214.9094062 linkify_it_py-2.2.0/linkify_it_py.egg-info/top_level.txt0000644000175100017510000000001315244502417023125 0ustar00runnerrunnerlinkify_it ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987211.2111893 linkify_it_py-2.2.0/pyproject.toml0000644000175100017510000000306115244502413016746 0ustar00runnerrunner[build-system] requires = ["setuptools"] build-backend = "setuptools.build_meta" [project] name = "linkify-it-py" authors = [{ name = "tsutsu3" }] description = "Links recognition library with FULL unicode support." urls = { Homepage = "https://github.com/tsutsu3/linkify-it-py" } readme = "README.md" requires-python = ">=3.10" keywords = ["linkify", "linkifier", "autolink", "autolinker"] license = { text = "MIT" } classifiers = [ "Development Status :: 5 - Production/Stable", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.14", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Intended Audience :: Developers", "Topic :: Software Development :: Libraries :: Python Modules", ] dynamic = ["version"] [project.optional-dependencies] test = ["pytest", "coverage", "pytest-cov", "pytest-timeout"] dev = ["pre-commit", "isort", "flake8", "black", "pyproject-flake8"] benchmark = ["pytest", "pytest-benchmark"] doc = ["sphinx", "sphinx_book_theme", "myst-parser"] [tool.setuptools] packages = ["linkify_it"] [tool.setuptools.dynamic] version = { attr = "linkify_it.__version__" } [tool.isort] multi_line_output = 3 include_trailing_comma = true force_grid_wrap = 0 use_parentheses = true ensure_newline_before_comments = true line_length = 88 [tool.flake8] max-line-length = 88 extend-ignore = "E203, W503" ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987214.9260495 linkify_it_py-2.2.0/setup.cfg0000644000175100017510000000004615244502417015657 0ustar00runnerrunner[egg_info] tag_build = tag_date = 0 ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987214.9241197 linkify_it_py-2.2.0/test/0000755000175100017510000000000015244502417015015 5ustar00runnerrunner././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987211.2111893 linkify_it_py-2.2.0/test/__init__.py0000644000175100017510000000000015244502413017110 0ustar00runnerrunner././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1787987214.9246166 linkify_it_py-2.2.0/test/fixtures/0000755000175100017510000000000015244502417016666 5ustar00runnerrunner././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1787987211.211875 linkify_it_py-2.2.0/test/fixtures/links.txt0000644000175100017510000001453415244502413020552 0ustar00runnerrunner% % Regular links % My http://example.com site http://example.com My http://example.com/ site http://example.com/ http://example.com/foo_bar/ http://user:pass@example.com:8080 http://user@example.com http://user@example.com:8080 http://user:pass@example.com [https](https://www.ibm.com)[mailto](mailto:someone@ibm.com) % should not catch as auth (before @ in big link) https://www.ibm.com http://example.com:8080 http://example.com/?foo=bar http://example.com?foo=bar http://example.com/#foo=bar http://example.com#foo=bar http://a.in HTTP://GOOGLE.COM http://example.invalid % don't restrict root domain when schema exists http://example.invalid http://inrgess2 % Allow local domains to end with digit http://inrgess2 http://999 % ..and start with digit, and have digits only http://999 http://host-name % local domain with dash http://host-name >>example.com % markdown blockquote example.com >>http://example.com % markdown blockquote http://example.com http://lyricstranslate.com/en/someone-you-നിന്നെ-പോലൊരാള്‍.html % With control character http://lyricstranslate.com/en/someone-you-നിന്നെ-പോലൊരാള്‍.html % % localhost (only with protocol allowed) % //localhost //test.123 http://localhost:8000? http://localhost:8000 % % Other protocols % My ssl https://example.com site https://example.com My ftp://example.com site ftp://example.com % % Neutral proto % My ssl //example.com site //example.com % % IPs % 4.4.4.4 192.168.1.1/abc % % Fuzzy % test.example@http://vk.com http://vk.com text:http://example.com/ http://example.com/ google.com google.com: // no port google.com s.l.o.w.io a-b.com GOOGLE.COM. GOOGLE.COM google.xxx // known tld google.xxx % % Correct termination for . , ! ? [] {} () "" '' % (Scoped http://example.com/foo_bar) http://example.com/foo_bar http://example.com/foo_bar_(wiki) http://foo.com/blah_blah_[other] http://foo.com/blah_blah_{I'm_king} http://foo.com/blah_blah_I'm_king http://www.kmart.com/bestway-10'-x-30inch-steel-pro-frame-pool/p-004W007538417001P http://foo.com/blah_blah_"doublequoted" http://foo.com/blah_blah_'singlequoted' (Scoped like http://example.com/foo_bar) http://example.com/foo_bar [Scoped like http://example.com/foo_bar] http://example.com/foo_bar {Scoped like http://example.com/foo_bar} http://example.com/foo_bar "Quoted like http://example.com/foo_bar" http://example.com/foo_bar 'Quoted like http://example.com/foo_bar' http://example.com/foo_bar [example.com/foo_bar.jpg)] example.com/foo_bar.jpg http://example.com/foo_bar.jpg. http://example.com/foo_bar.jpg http://example.com/foo_bar/. http://example.com/foo_bar/ http://example.com/foo_bar, http://example.com/foo_bar http://index-of.es/Android/Professional.Android.2.Application.Development.(Wrox,.2010,.0470565527).pdf https://github.com/markdown-it/linkify-it/compare/360b13a733f521a8d4903d3a5e1e46c357e9d3ce...f580766349525150a80a32987bb47c2d592efc33 https://www.google.com/search?sxsrf=ACYBGNTJFmX-GjNJ8fM-2LCkqyNyxGU1Ng%3A1575534146332&ei=Qr7oXf7rE4rRrgSEgrmoAw&q=clover&oq=clover&gs_l=psy-ab.3..0i67j0l9.2986.3947..4187...0.2..0.281.1366.1j0j5......0....1..gws-wiz.......0i71j35i39j0i131.qWp1nz4IJVA&ved=0ahUKEwj-lP6Iip7mAhWKqIsKHQRBDjUQ4dUDCAs&uact=5 https://ourworldindata.org/grapher/covid-deaths-days-since-per-million?zoomToSelection=true&time=9..&country=FRA+DEU+ITA+ESP+GBR+USA+CAN http://example.com/foo_bar... http://example.com/foo_bar http://172.26.142.48/viewerjs/#../0529/slides.pdf http://example.com/foo_bar.. http://example.com/foo_bar http://example.com/foo_bar?p=10. http://example.com/foo_bar?p=10 https://www.google.ru/maps/@59.9393895,30.3165389,15z?hl=ru https://www.google.com/maps/place/New+York,+NY,+USA/@40.702271,-73.9968471,11z/data=!4m2!3m1!1s0x89c24fa5d33f083b:0xc80b8f06e177fe62?hl=en https://www.google.com/analytics/web/?hl=ru&pli=1#report/visitors-overview/a26895874w20458057p96934174/ http://business.timesonline.co.uk/article/0,,9065-2473189,00.html https://google.com/mail/u/0/#label/!!!Today/15c9b8193da01e65 http://example.com/123! http://example.com/123 http://example.com/123!!! http://example.com/123 http://example.com/foo--bar See http://example.com/123; Example link. http://example.com/123 http://example.com/123;123 % some sites have links with trailing dashes http://www.bloomberg.com/news/articles/2015-06-26/from-deutsche-bank-to-siemens-what-s-troubling-germany-inc- http://example.com/foo-with-trailing-dash-dot-. http://example.com/foo-with-trailing-dash-dot- http://domain.com . http://domain.com http://domain.com/foo . http://domain.com/foo domain.com . domain.com domain.com/foo user@domain.com . user@domain.com mailto:user@domain.com % % Emails % test."foo".bar@gmail.co.uk! test."foo".bar@gmail.co.uk "test@example.com" test@example.com name@example.com >>name@example.com % markdown blockquote name@example.com mailto:name@example.com MAILTO:NAME@EXAMPLE.COM mailto:foo_bar@example.com foo+bar@gmail.com 192.168.1.1@gmail.com mailto:foo@bar % explicit protocol make it valid mailto:foo@bar (foobar email@example.com) email@example.com (email@example.com foobar) email@example.com (email@example.com) email@example.com % % International % http://✪df.ws/123 http://xn--df-oiy.ws/123 a.ws ➡.ws/䨹 example.com/䨹 президент.рф % Links below provided by diaspora* guys, to make sure regressions will not happen. % Those left here for historic reasons. http://www.bürgerentscheid-krankenhäuser.de http://www.xn--brgerentscheid-krankenhuser-xkc78d.de http://bündnis-für-krankenhäuser.de/wp-content/uploads/2011/11/cropped-logohp.jpg http://xn--bndnis-fr-krankenhuser-i5b27cha.de/wp-content/uploads/2011/11/cropped-logohp.jpg http://ﻡﻮﻘﻋ.ﻭﺯﺍﺭﺓ-ﺍﻼﺘﺻﺍﻼﺗ.ﻢﺻﺭ/ http://xn--4gbrim.xn----ymcbaaajlc6dj7bxne2c.xn--wgbh1c/ % % Others... % ｜www.google.com/www.google.com/foo｜bar % #46, asian vertical pipes www.google.com/www.google.com/foo ｜test@google.com｜bar test@google.com ｜http://google.com｜bar http://google.com % % Domains with multiple dashes % https://5b0ee223b312746c1659db3f--thelounge-chat.netlify.com/docs/ www.a--b.com www.c--u.com http://a---b.com/ ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1787987211.211996 linkify_it_py-2.2.0/test/fixtures/not_links.txt0000644000175100017510000000137315244502413021427 0ustar00runnerrunner% % Not links % example.invalid example.invalid/ http://.example.com http://-example.com hppt://example.com example.coma -example.coma foo.123 localhost % only with protocol allowed localhost/ ///localhost % 3 '/' not allowed ///test.com //test % Don't allow single level protocol-less domains to avoid false positives _http://example.com _//example.com _example.com http://example.com_ @example.com node.js and io.js http:// http://. http://.. http://# http://## http://? http://?? google.com:500000 // invalid port show image.jpg path:to:file.pm /path/to/file.pl % % Not IPv4 % 1.2.3.4.5 1.2.3 1.2.3.400 1000.2.3.4 a1.2.3.4 1.2.3.4a % % Not email % foo@bar % Should be at second level domain & with correct tld mailto:bar ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1787987211.211996 linkify_it_py-2.2.0/test/fixtures/port_links.txt0000644000175100017510000000265515244502413021617 0ustar00runnerrunner% % Cases that fail only in this port, not in upstream linkify-it. % % % `;` in the email name. Upstream's src_email_name has allowed it since 5.0.0, % but this port dropped it from both halves of the character class. % a;b@example.com foo;bar;baz@example.com ;a@example.com a;@example.com mailto:a;b@example.com % % Astral-plane characters in host names. uc-micro-py's classes are UTF-16 % surrogate-pair regexes from JavaScript, so they never fire on Python str and % these hosts are not recognized. linkify-it handles them. % 𠮷.com 𠮷野家.com 𝔞𝔟.com example@𠮷.com % % Punctuation terminates a host. These are BMP controls: they already pass and % agree with linkify-it, and they are what makes the astral expectation in the % next block the consistent one. % a、b.com b.com a·b.com b.com a።b.com b.com % =========================================================================== % Deliberate divergence from linkify-it % =========================================================================== % % Astral punctuation terminating a host. linkify-it returns the whole % "a𐩐b.com": there U+10A50 is two UTF-16 code units and only the first is % recognized as punctuation, so the second slips through as a letter. Python % has no such split, so we terminate the host exactly like the BMP controls % above. Expect these to show up as differences when diffing against upstream. % a𐩐b.com b.com a𑁍b.com b.com ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1787987211.211996 linkify_it_py-2.2.0/test/test_apis.py0000644000175100017510000002217715244502413017367 0ustar00runnerrunnerimport re import pytest from linkify_it import LinkifyIt, SchemaError from linkify_it.main import Match from linkify_it.tlds import TLDS def test_pretest_false(): linkifyit = LinkifyIt() assert linkifyit.pretest("nolink") is False def test_create_instance_with_schemas(): schemas = {"my:": {"validate": r"^\/\/[a-z]+"}} linkifyit = LinkifyIt(schemas) match = linkifyit.match("google.com. my:// my://asdf!") assert match[0].text == "google.com" assert match[1].text == "my://asdf" def test_match_class(): match = Match("", "", -1, -1) assert ( match.__repr__() == "linkify_it.main.Match({'schema': '', 'index': -1, 'last_index': -1, 'raw': '', 'text': '', 'url': ''})" # noqa: E501 ) def test_api_extend_tlds(): linkifyit = LinkifyIt() assert linkifyit.test("google.myroot") is False linkifyit.tlds("myroot", True) assert linkifyit.test("google.myroot") is True assert linkifyit.test("google.xyz") is False # ref - http://data.iana.org/TLD/tlds-alpha-by-domain.txt linkifyit.tlds(TLDS) assert linkifyit.test("google.xyz") is True assert linkifyit.test("google.myroot") is False def test_api_add_rule_as_regex_with_default_normalizer(): linkifyit = LinkifyIt().add("my:", {"validate": re.compile(r"^\/\/[a-z]+")}) match = linkifyit.match("google.com. my:// my://asdf!") assert match[0].text == "google.com" assert match[1].text == "my://asdf" def test_api_add_rule_as_regex_with_default_normalizer_with_no_compile(): linkifyit = LinkifyIt().add("my:", {"validate": r"^\/\/[a-z]+"}) match = linkifyit.match("google.com. my:// my://asdf!") assert match[0].text == "google.com" assert match[1].text == "my://asdf" def test_api_add_rule_with_normalizer(): def func_normalize(self, m): m.text = re.sub(r"^my://", "", m.text).upper() m.url = m.url.upper() linkifyit = LinkifyIt().add( "my:", {"validate": re.compile(r"^\/\/[a-z]+"), "normalize": func_normalize} ) match = linkifyit.match("google.com. my:// my://asdf!") assert match[1].text == "ASDF" assert match[1].url == "MY://ASDF" def test_api_add_rule_with_normalizer_no_cimpile(): def func_normalize(self, m): m.text = re.sub(r"^my://", "", m.text).upper() m.url = m.url.upper() linkifyit = LinkifyIt().add( "my:", {"validate": r"^\/\/[a-z]+", "normalize": func_normalize} ) match = linkifyit.match("google.com. my:// my://asdf!") assert match[1].text == "ASDF" assert match[1].url == "MY://ASDF" def test_api_disable_rule(): linkifyit = LinkifyIt() assert linkifyit.test("http://google.com") assert linkifyit.test("foo@bar.com") linkifyit.add("http:", None) linkifyit.add("mailto:", None) assert not linkifyit.test("http://google.com") assert not linkifyit.test("foo@bar.com") def test_api_add_bad_definition(): with pytest.raises(SchemaError): linkifyit = LinkifyIt({"fuzzy_link": False}) linkifyit = LinkifyIt() with pytest.raises(SchemaError): linkifyit.add("test:", []) linkifyit = LinkifyIt() with pytest.raises(SchemaError): linkifyit.add("test:", {"validate": []}) linkifyit = LinkifyIt() with pytest.raises(SchemaError): def func(): return False linkifyit.add("test:", {"validate": func, "normalize": "bad"}) def test_api_at_position(): linkifyit = LinkifyIt() assert linkifyit.test_schema_at("http://google.com", "http:", 5) assert linkifyit.test_schema_at("http://google.com", "HTTP:", 5) assert not linkifyit.test_schema_at("http://google.com", "http:", 6) assert not linkifyit.test_schema_at("http://google.com", "bad_schema:", 6) def test_api_correct_cache_value(): linkifyit = LinkifyIt() match = linkifyit.match(".com. http://google.com google.com ftp://google.com") assert match[0].text == "http://google.com" assert match[1].text == "google.com" assert match[2].text == "ftp://google.com" def test_api_normalize(): linkifyit = LinkifyIt() match = linkifyit.match("mailto:foo@bar.com")[0] # assert match.text == "foo@bar.com" assert match.url == "mailto:foo@bar.com" match = linkifyit.match("foo@bar.com")[0] # assert match.text == "foo@bar.com" assert match.url == "mailto:foo@bar.com" def test_api_twitter_rule(): linkifyit = LinkifyIt() def validate(self, text, pos): tail = text[pos:] if not self.re.get("twitter"): self.re["twitter"] = re.compile( "^([a-zA-Z0-9_]){1,15}(?!_)(?=$|" + self.re["src_ZPCc"] + ")" ) if self.re["twitter"].search(tail): if pos > 2 and tail[pos - 2] == "@": return False return len(self.re["twitter"].search(tail).group()) return 0 def normalize(self, m): m.url = "https://twitter.com/" + re.sub(r"^@", "", m.url) linkifyit.add("@", {"validate": validate, "normalize": normalize}) assert linkifyit.match("hello, @gamajoba_!")[0].text == "@gamajoba_" assert linkifyit.match(":@givi")[0].text == "@givi" assert linkifyit.match(":@givi")[0].url == "https://twitter.com/givi" assert not linkifyit.test("@@invalid") def test_api_twitter_rule_no_compile(): linkifyit = LinkifyIt() def validate(self, text, pos): tail = text[pos:] if not self.re.get("twitter"): self.re["twitter"] = ( "^([a-zA-Z0-9_]){1,15}(?!_)(?=$|" + self.re["src_ZPCc"] + ")" ) if re.search(self.re["twitter"], tail): if pos > 2 and tail[pos - 2] == "@": return False return len(re.search(self.re["twitter"], tail).group()) return 0 def normalize(self, m): m.url = "https://twitter.com/" + re.sub(r"^@", "", m.url) linkifyit.add("@", {"validate": validate, "normalize": normalize}) assert linkifyit.match("hello, @gamajoba_!")[0].text == "@gamajoba_" assert linkifyit.match(":@givi")[0].text == "@givi" assert linkifyit.match(":@givi")[0].url == "https://twitter.com/givi" assert not linkifyit.test("@@invalid") def test_api_set_option_fuzzylink(): linkifyit = LinkifyIt(options={"fuzzy_link": False}) assert not linkifyit.test("google.com") linkifyit.set({"fuzzy_link": True}) assert linkifyit.test("google.com") assert linkifyit.match("google.com")[0].text == "google.com" def test_api_set_option_fuzzyemail(): linkifyit = LinkifyIt(options={"fuzzy_email": False}) assert not linkifyit.test("foo@bar.com") linkifyit.set({"fuzzy_email": True}) assert linkifyit.test("foo@bar.com") assert linkifyit.match("foo@bar.com")[0].text == "foo@bar.com" def test_api_set_option_fuzzyip(): linkifyit = LinkifyIt() assert not linkifyit.test("1.1.1.1") linkifyit.set({"fuzzy_ip": True}) assert linkifyit.test("1.1.1.1") assert linkifyit.match("1.1.1.1")[0].text == "1.1.1.1" def test_api_shoud_not_hang_in_fuzzy_mode_with_sequence_of_astrals(): linkifyit = LinkifyIt() linkifyit.set({"fuzzy_link": True}) linkifyit.match( "😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡😡 .com" ) def test_api_shoud_accept_triple_minus(): linkifyit = LinkifyIt() assert linkifyit.match("http://e.com/foo---bar")[0].text == "http://e.com/foo---bar" assert linkifyit.match("text@example.com---foo") is None linkifyit = LinkifyIt(None, {"---": True}) assert linkifyit.match("http://e.com/foo---bar")[0].text == "http://e.com/foo" assert linkifyit.match("text@example.com---foo")[0].text == "text@example.com" # issue #25. Schema key containing - not producing matches @pytest.mark.parametrize( "escape_str", {".", "?", "*", "+", "^", "$", "[", "]", "\\", "(", ")", "{", "}", "|", "-"}, ) def test_api_add_alias_rule_with_excape_re_string(escape_str): linkifyit = LinkifyIt() linkifyit.add(f"foo{escape_str}bar:", "http:") assert linkifyit.test(f"Check foo{escape_str}bar://test") is True def test_api_blank_test_match_at_the_start(): linkifyit = LinkifyIt() assert not linkifyit.match_at_start("") def test_api_should_find_a_match_at_the_start(): linkifyit = LinkifyIt() linkifyit = LinkifyIt(options={"fuzzy_link": True}) linkifyit.set({"fuzzy_link": True}) assert not linkifyit.test("@@invalid") assert not linkifyit.match_at_start("google.com 123") assert not linkifyit.match_at_start(" http://google.com 123") def test_api_match_a_start_should_not_interfere_with_normal_match(): linkifyit = LinkifyIt() str = "http://google.com http://google.com" assert linkifyit.match_at_start(str) assert len(linkifyit.match(str)) == 2 str = "aaa http://google.com http://google.com" assert not linkifyit.match_at_start(str) assert len(linkifyit.match(str)) == 2 def test_api_should_not_match_incomplete_links(): # regression test for https://github.com/markdown-it/markdown-it/issues/868 linkifyit = LinkifyIt() assert not linkifyit.match_at_start("http://") assert not linkifyit.match_at_start("https://") ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1787987211.211996 linkify_it_py-2.2.0/test/test_linkify.py0000644000175100017510000000157215244502413020074 0ustar00runnerrunnerfrom pathlib import Path import pytest from linkify_it import LinkifyIt from .utils import read_fixture_file FIXTURE_PATH = Path(__file__).parent / "fixtures" def dummy(_): pass @pytest.mark.parametrize( "number,line,expected", read_fixture_file(FIXTURE_PATH.joinpath("links.txt")), ) def test_links(number, line, expected): linkifyit = LinkifyIt(options={"fuzzy_ip": True}) linkifyit.normalize = dummy assert linkifyit.pretest(line) is True assert linkifyit.test("\n" + line + "\n") is True assert linkifyit.test(line) is True assert linkifyit.match(line)[0].url == expected @pytest.mark.parametrize( "number,line,expected", read_fixture_file(FIXTURE_PATH.joinpath("not_links.txt")), ) def test_not_links(number, line, expected): linkifyit = LinkifyIt() linkifyit.normalize = dummy assert linkifyit.test(line) is False ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1787987211.211996 linkify_it_py-2.2.0/test/test_pathological.py0000644000175100017510000000154315244502413021073 0ustar00runnerrunner"""Inputs that used to make the scan loop degrade to O(n^2).""" import pytest from linkify_it import LinkifyIt TIMEOUT_SECONDS = 10 @pytest.mark.timeout(TIMEOUT_SECONDS) def test_should_not_hang_on_fuzzy_links_followed_by_an_email_marker(): # ~1 MiB input LinkifyIt().match("a.com " * 174762 + "@") @pytest.mark.timeout(TIMEOUT_SECONDS) def test_should_not_hang_on_fuzzy_emails_followed_by_a_link_marker(): # ~1 MiB input LinkifyIt().match("a@b.com " * 131071 + ".com") @pytest.mark.timeout(TIMEOUT_SECONDS) def test_should_not_hang_on_repeated_mailto_schema_prefixes(): # ~680 KiB input. The ":" in mailto's prefix is also a valid email-name # char, so "mailto:mailto:..." chains into O(n) schema hits, each running # the mailto validator to the end of the tail => O(n^2) without a fix. LinkifyIt().match("mailto:" * 100000) ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1787987211.211996 linkify_it_py-2.2.0/test/test_port_parity.py0000644000175100017510000000152515244502413021001 0ustar00runnerrunner"""Links that linkify-it recognizes but this port did not, due to port bugs. Kept apart from test_linkify.py so that upstream's own fixtures stay byte identical with linkify-it and remain diffable on the next port. """ from pathlib import Path import pytest from linkify_it import LinkifyIt from .utils import read_fixture_file FIXTURE_PATH = Path(__file__).parent / "fixtures" def dummy(_): pass @pytest.mark.parametrize( "number,line,expected", read_fixture_file(FIXTURE_PATH.joinpath("port_links.txt")), ) def test_port_links(number, line, expected): linkifyit = LinkifyIt(options={"fuzzy_ip": True}) linkifyit.normalize = dummy assert linkifyit.pretest(line) is True assert linkifyit.test("\n" + line + "\n") is True assert linkifyit.test(line) is True assert linkifyit.match(line)[0].url == expected ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1787987211.211996 linkify_it_py-2.2.0/test/utils.py0000644000175100017510000000141615244502413016525 0ustar00runnerrunnerimport re def lget(src_list, index): try: return src_list[index] except IndexError: return "" def read_fixture_file(path): tests = [] skip_next = False comment_re = re.compile(r"^%.*") with open(path, encoding="utf-8") as f: lines = [x.rstrip() for x in f.readlines()] for idx, line in enumerate(lines): if skip_next: skip_next = False continue line = comment_re.sub("", line) next_line = comment_re.sub("", lget(lines, idx + 1)) if not line.strip(): continue if next_line.strip(): tests.append([idx + 1, line, next_line]) skip_next = True else: tests.append([idx + 1, line, line]) return tests ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1787987211.211996 linkify_it_py-2.2.0/tox.ini0000644000175100017510000000021415244502413015342 0ustar00runnerrunner[tox] envlist = py{310-314} [testenv] deps = pytest usedevelop = true [testenv:py{310-314}] extras = test commands = pytest {posargs}