Asked Python experts what useful Python libraries they use and recommend for other developers.
Python is an untyped programming language and on the one hand, this is its advantage, which facilitates quick learning and sets a low entry threshold, but at some point, the lack of typing becomes a problem.
With pydantic and PEP 484 datatype hints, it is possible to turn python into a strongly typed language. Especially useful when your application has APIs but no contracts.
Convenient logger to replace the standard logging. In addition to the fact that it does not require complex configurations, it has a huge number of useful features and works well with asynchronous code.
Colorful output of logs to the console, informative tracebacks with different levels and tips, a convenient built-in log parser, multithreading support and thread safety – this is a small part of the functionality of this excellent Python library. In terms of ease of use, it is comparable to a regular print, but in terms of functionality, it is a whole rocket.
If you cover your code with unit tests, then you are familiar with the situation where you spend time generating test data and various fixtures. This is especially true when you are using ORM and you have many models with many fields.
Instead of spending time writing various datasets and generator helpers to cover all possible cases, it is enough to create a factory, specify the model under test, and specify the desired number of fixtures when called.
An excellent Python library from Yandex for natural language processing. Unlike pymorphy2, it does not bring the word to normal form so well, but it is excellent, and most importantly – very quickly, it determines the part of speech and lexemes of the word. Works only with Russian.
In parallel with Python, the PEP8 coding convention has evolved. Codestyle is an essential part of any project, especially when working in a team. It takes a lot of time and practice to follow all the canons, but you can use black to keep your code looking the way it should.
This autoformatter will save you time, reduce the number of comments on reviews and your code will always be the same, regardless of the project.
In Python, we mainly develop solutions in the field of computer vision (for example, in such ), so our collection is somehow related to this. They help a lot:
- Flask is a micro web framework that allows you to quickly create services and integrate our solutions. Has many useful extensions. Suitable for both experimentation and industrial use.
- Keras is a cool high-level API for TensorFlow. Allows you to save a lot of time and keep the code readable. Quite low entry threshold compared to pure TF.
- Numpy is a very handy tool for working with multidimensional arrays and matrices, indispensable in deep learning.
- Pillow is a good old Python library for working with images. Many formats, pixel manipulations, filters, effects.
Sometimes it becomes necessary to write down Cyrillic words in Latin letters. There are different standards and rules for transliteration, so somewhere I am Dmitrij, somewhere Dmitry, and somewhere Dmitrii.
In the library, the standards are brought together and the special rules for some combinations of letters and word endings are correctly implemented. The home page summarizes the differences between the diagrams for easy selection.
In almost any project, you need to receive data from users or from colleagues from other projects.
This data must be validated so that it will not be excruciatingly painful later. The pydantic project allows you to get rid of the routine and make validation comfortably for different data types. You can describe the required data structure in order not only to check them at the input, but also to receive detailed messages if there are errors. You can write your own validation rules for individual fields, validate some fields together (for example, if the last name field is filled in, then the name field must also be filled in) and much more.
I think almost every pythonist has come across the Celery project.
Dramatiq is a project of a person who is so tired of some of the problems at Celery that he decided to write his own project. It turned out very well. If you need to perform any tasks that you want to send to the background (recalculation of some indicators, mass updating of various data from external services, preparing letters for mailing, generating reports, etc.) this library is for you.
In the Python world, the requests library is practically the benchmark for usability and functionality. Unfortunately, it cannot be used in asynchronous code.
The authors of httpx create an asynchronous library for working with http with the same interface as requests. You can also work synchronously with httpx.
The project is still officially in beta, but it is already quite stable and is used by many programmers.
Configuring logging through the standard logging library is not fun. If your project has not grown enough to set up a syslog, journald, or ELK stack for it, but you already want to see good logs, try loguru.
The main advantage of this Python library is its ease of customization and a bunch of nice goodies out of the box:
- highlighting messages in different colors,
- good formatting,
- convenient configuration of logging to files with rotation and archiving, and much more.
A life-saving library that will come to the rescue when you need to recognize a date from strings of various formats. It will be useful if you are parsing web pages or some kind of logs from a variety of sources.
It can recognize both ordinary variants, for example, ‘March 2, 2021 at 15:00’, ‘2021-03-04 10:01:02 UTC + 3’, and such extreme as ‘in 5 days’, ‘a week ago’. Supports many languages.
Sometimes you need to run a script that processes a lot of data (it could be a management command for Django or just a script for converting a large number of files). If nothing happens in the console, then after a while it seems that the script is frozen, it is not clear at what speed the process is progressing, how long to wait.
Using the tqdm library, you can quickly create indicators (progress bars) to display the work progress. Integrates with IPython / Jupyter out of the box.
In my opinion, this is one of the most useful tools when it becomes necessary to measure, track and analyze the memory usage of specific objects in python applications.
A handy Python library that allows you to repeat a function call if it has not been successfully completed. For example, this can be useful when it is necessary to repeat the request to some external service, if the first time it was not possible to “reach out” to it. Of course, you can change many variables, such as the number of attempts, the time to wait for attempts, and so on.
A tool for simulating responses to requests made using the requests library . It is very useful for testing functional behavior checking depending on what kind of response the external service will return.
Faker allows you to generate dummy data in various categories. For example, names, addresses, bank card details, phone numbers, and so on. The list of categories is really extensive. Quite often, this particular library is necessary when testing functionality.
The tool provides great features and a convenient API for formatting the text output by the application to the console. Allows you to change the style of text and its color, draw tables, highlight the syntax of the language, work with emoji and much more. I want to draw your attention to the fact that the library is not written in python, but helps in working with python applications.
Ry-spy is a sampling profiler for Python applications. A distinctive feature of this profiler is that it allows you to connect to an already running application that needs to be analyzed without adding any code. The tool provides the ability to monitor in real time what is currently being performed and how long it takes, accumulate information about the running application and use this information for, for example, generating a flamegraph . It is very useful when you need to find a problem in the application “here and now”, or when it is not possible to change the code to integrate with other profilers.
There are situations when you need to write tests for functionality that behaves differently depending on the current time / date. This tool is just for such cases. It allows you to fix the time at the value you need by replacing the datetime module.
Also a good library that contains a set of functional utilities.