Data Warehouses Meet Data Lakes

uk: Комп'ютерні науки

description: EuroPython 2022 - Data Warehouses Meet Data Lakes - presented by Mauro [Liffey Hall 1 on 2022-07-13] In this talk, I will explain the current challenges of a datalake and how we can approach a moderm data architecture with the help of pyspark, hudi, delta.io or iceberg. We will see how organize data in a data lake to support real-time processing of applications and analyzes across all varieties of data sets, structured and unstructured, how provides the scale needed to support enterprise-wide digital transformation and creates one unique source of data for multiple audiences. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-nc-sa/4.0/

uk: вхід без авторизації


Andrii Soldatenko

type: Person

cs: Informatika

de: Informatik

fi: tietojenkäsittelytiede

uk: Комп'ютерні науки

en: Computer Science

fr: Informatique

da: Datalogi

es: Ciencias de la Computación

nl: Informatica

cs: Technické vědy

de: Ingenieurwissenschaften

fi: tekniikka

uk: Інженерія

en: Engineering Sciences

fr: Ingénierie

da: Ingeniørvidenskab

es: Ingeniería

nl: Engineering

description: Luckily, when it comes to debugging asynchronous applications in python, we have a couple of options to consider. The writers of the asyncio module have very kindly provided a debug mode, which is quite powerful and can really aid us in our debugging adventures without the overhead of modifying the system's code base too dramatically. In particular, I'll show you what asyncio debug mode means for developers, and how to source tracebacks for unhandled exceptions in futures. Also how to detect accidental blocking for I/O. We discuss how to monitor the asyncio event loop and collect metrics in statsD. Finally, we discuss monitor and cli capabilities for asyncio applications based on aiomonitor and aioconsole. Also, we discuss how asynchronous python works in REPL via autoawait based on ipython and how to do it in vanilla python REPL.

TIB AV-Portal

type: Service

type: WebContent

type: MediaObject

@language: en

cs: přihlášení není vyžadováno

de: keine Anmeldung erforderlich

uk: вхід без авторизації

en: no login required

nl: geen login vereist

name: Debugging asynchronous programs in Python

EuroPython Conference

type: Organization

cs: Video

de: Video

fi: Video

uk: Відео

en: Video

fr: Vidéo

da: Video

nl: Video

es: Video


Demystifying Python’s Internals

uk: Комп'ютерні науки

description: EuroPython 2022 - Demystifying Python’s Internals: Diving into CPython by implementing a pipe operator - presented by Sebastiaan Zeeff [The Auditorium on 2022-07-15] Diving into the CPython source code can feel daunting. Whether you want to start contributing or just want to get a better understanding of Python by exploring its source code, it’s often difficult to know where to start or what you’re missing. In my talk, I will show you around the CPython source code by implementing a new operator, a pipe operator. While doing so, I will discuss core parts of the internals, such as Python’s grammar, its syntax trees, and the underlying logic that will perform the operation. By the end, you will have a good idea of the moving parts involved in core language features. I will also take you through the steps necessary to make it all work. I’ll show you how I obtained a copy of the source code, regenerated the parser and token files, and how I compiled my modified version of CPython. I will also write and run tests to help me implement my changes. This should give you a mental framework that helps you while diving into more comprehensive resources, like the excellent Python Developer’s Guide (https://devguide.python.org/). My talk is aimed at everyone who wants to explore CPython’s internals. You don’t have to be an expert in Python, although some affinity with Python helps with understanding the internals. I will also use C to implement some of the operator logic, but knowledge of C is by no means required. In short, if you’re interested in diving into the CPython source code, this talk is for you. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-nc-sa/4.0/

uk: вхід без авторизації


uk: Комп'ютерні науки

description: You can have the best product in your expertise area, but if your documentation isn’t on par with the flawless experience you want to offer to the world, success is not guaranteed. Let’s be real here: documentation is often an afterthought and rarely included in life cycle development processes. Still, documentation is the secret weapon for greater adoption, and growth that you may have not known you could achieve. It’s time for you to step up your game and measure up to the big players. Learn about the benefits of high quality and educational documentation and the true role it plays in the developer community. You’ll also learn the principles of a solid foundation, and tips on how to use one of the most powerful developer relations’ tools.

uk: вхід без авторизації

name: Developers Documentation: your secret weapon


uk: Комп'ютерні науки

description: EuroPython 2022 - Diversity & Inclusion in the Python Community Panel - presented by Tereza Iofciu, Marlene Mhangami, Naomi Ceder, Nabanita Roy & Iqbal Abdullah [The Auditorium on 2022-07-14] Come meet some of the folks working on Diversity and Inclusion in the global Python Community! Join the live panel discussion to hear about the challenges and the work they do. With Marlene Mhangami, Nabanita Roy, Iqbal Abdullah, Tereza Iofciu. Chaired by Naomi Ceder. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-nc-sa/4.0/

uk: вхід без авторизації


uk: Комп'ютерні науки

description: You've been a developer for a couple of years already. Your journey started as an intern/junior-level position where you were learning to code; then, going through mid and senior positions, you were offered the team lead or even engineering manager role. At this moment, you have plenty of questions and doubts. How to answer them to make sure you make a good decision? How to prepare for the new role if you want to take it? I'll help you answer these questions and prepare for your future role."

uk: вхід без авторизації

name: Dr. Jekyll & Mr. Hyde - transition from developer to manager


Education Panel

uk: Комп'ютерні науки

description: Teaching Python: a panel discussion with perspectives from teachers, academics, makers and enthusiasts. Why is Python appealing in education? What tools and resources work well? What can the Python community do to help teachers & policy makers? Join us for an engaging and insightful discussion with a fascinating panel of experts, Dr Keith Quille, Kelly Schuster-Paredes, Chris Reina and Sarah-Jayne Carey.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: A few weeks ago I was working on setting up a relational database to explore records from DataSF’s Civic Art Collection. Whenever I attend a tech conference I try to spend a day or two in the city to check out its cultural scene, so this seemed like useful information! I decided to use MySQL as my database engine. Coming from a Pandas background I was surprised by how unproductive and restricted I felt writing raw SQL queries. I also spent a significant amount of time resolving errors in queries that worked with one flavor of SQL but failed with MySQL. Throughout the process, I kept thinking to myself if only there was a more Pythonic way!!! A few weeks later I was introduced to Ibis. I live in Zimbabwe and the first thing that pops into my mind when I think of the word ibis is a safari. One of my favorite things to do when I'm not working is to go on a game drive. Whenever I've been adventuring on safari I usually see ibises perched on top of an elephant. The contrast between the creatures is stark! The African Sacred Ibis is a small, elegant creature that's named after the ancient Egyptian god Thoth. While as many of us know, an elephant is a very big and complex animal. This image serves as a great metaphor for the Python package and how it interacts with big database engines. Ibis allows you to write intuitive Python code and have that code be translated into SQL. Whether you’re wanting to interact with SQL databases or wanting to use distributed DBMSs, Ibis lets you do this in Python. You can think of the python code as the less complex elegant layer sitting on top of any big data engine of your choice. At the moment, Ibis supports quite a few backends including: Traditional DBMSs: PostgreSQL, MySQL, SQLite Analytical DBMSs: OmniSciDB, ClickHouse, Datafusion Distributed DBMSs: Impala, PySpark, BigQuery In memory analytics: pandas and Dask. Anything you can write in an SQL select statement you can write in Ibis. You can carry out joins, filters, and other operations on your data in a familiar, Pandas-like syntax. In this talk, we'll go through several examples of these and compare what the SQL code would look like versus writing to the database with Ibis. Overall, using Ibis simplifies your workflows, makes you more productive, and keeps your code readable.

uk: вхід без авторизації

name: Elephants, ibises and a more Pythonic way to work with databases


EuroPython 2022 Closing Session

uk: Комп'ютерні науки

description: EuroPython 2022 - EuroPython 2022 Closing Session [The Auditorium on 2022-07-15] See you at EuroPython 2023! This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-nc-sa/4.0/

uk: вхід без авторизації


EuroPython 2022 Opening Session

uk: Комп'ютерні науки

description: EuroPython 2022 Opening Session [The Auditorium on 2022-07-13] Welcome to EuroPython 2022! This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-nc-sa/4.0/

uk: вхід без авторизації


uk: Комп'ютерні науки

description: EuroPython 2022 - EuroPython 2022 Sponsor Highlight & Recruitment [Liffey A on 2022-07-14] Many of our sponsors are looking to hire talented people and EuroPython is the perfect place to reach out to them! In this session, our sponsors will each give a short presentation about their company and what they do with Python. You can then approach them directly at their booth to discuss more details. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-nc-sa/4.0/

uk: вхід без авторизації


Thursday's Lightning Talks

uk: Комп'ютерні науки

description: EuroPython 2022 - Thursday's Lightning Talks - presented by [The Auditorium on 2022-07-14] Signing up is on a first-come-first-served basis. The queue is reset every day in the morning. You may talk about / present (almost) everything with certain boundaries: - no promotion for products of companies, no call for 'we are hiring' (but you may name your employer). - conference announcements are limited to one minute only - one LT per person per conference policy: everyone who has not given an LT at this year's conference, yet, is ahead of you in the queue - keep it short, the shorter the presentations, the more people can give a talk See you there and have fun! This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-nc-sa/4.0/

uk: вхід без авторизації


uk: Комп'ютерні науки

description: Building distributed systems can be challenging, and that's what microservices are. With traditional request/response based architectures, it is easy to end up with services that are tightly coupled, making them difficult to maintain and extend independently. While not a silver bullet, an event-based architecture makes it easier to maintain loose coupling. It also makes it easier to extend and evolve our systems without resorting to a rewrite. In this presentation, we will look at an example of an event-driven microservices application and discuss some things to consider when adopting this approach.

uk: вхід без авторизації

name: Event-driven microservices with Python and Apache Kafka


uk: Комп'ютерні науки

description: The mono vs. multi-repo is an age-old debate in the DevOpsphere, and one that can still cause flame wars. What if I were to tell you that you don't have to choose? In this talk we will dive into how we built a centralized Git workflow that can work with any kind of repo architecture, delivered with Python. One of the greatest recurring pains in CI/CD is the need to reinvent the wheel and define your CI workflow for each and every repository or (micro)service, when eventually 99% of the config is the same. What if we could hard reset this paradigm and create a single, unified workflow that is shared by all of our repos and microservices? In this talk, we will showcase how a simple solution implemented in Python, demoed on Github as the SCM, and Github Actions for our CI, enabled us to unify this process for all of our services, and improve our CI/CD velocity by orders of magnitude.

uk: вхід без авторизації

name: Forget Mono vs. Multi-Repo - Building Centralized Git Workflows with Python


uk: Комп'ютерні науки

description: Signing up is on a first-come-first-served basis. The queue is reset every day in the morning. You may talk about / present (almost) everything with certain boundaries: - no promotion for products of companies, no call for 'we are hiring' (but you may name your employer). - conference announcements are limited to one minute only - one LT per person per conference policy: everyone who has not given an LT at this year's conference, yet, is ahead of you in the queue - keep it short, the shorter the presentations, the more people can give a talk

uk: вхід без авторизації

name: Friday Lightning Talks - EuroPython 2022


uk: Комп'ютерні науки

description: Ever had issues to manage your python packages and environment? Do you know how to create and share a package to the community? It can be challenging if you've never done it, but it also doesn't have to be hard. There is always a better tool to fit our needs. In this presentation, I'd like to discuss how Python's package managers appeared and evolved with time. Discussing pip, pipenv, and poetry, presenting each of their weak and strong points. Also intend to present how to package and publish a simple code with each one of them, and suggest which package manager should you choose, whether you are just starting with python, or feel like there is something bothering and never knew you could solve it easily and painless.

uk: вхід без авторизації

name: From pip to poetry - Python ways of packaging and publishing


Game Development with CircuitPython

uk: Комп'ютерні науки

description: Making games on small devices is great fun and also a great way to learn. You don't have to worry about breaking anything, and the games tend to be much simpler than on the big computers. And with CircuitPython you don't need to install anything on your computer, all you need is a text editor. But it's not easy to decide what you need exactly: what hardware to get, which libraries to use and how to actually put it all together to spend the minimum time on all that, and maximum time on the game itself? I'm going to go over the available options, their pros and cons, and show some examples to get you started.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: In the process of programming, we are always going to encounter various errors. Things rarely go as planned, especially in the world of programming. Errors are unavoidable when writing code, which can be frustrating at times. Every single one of us has faced this issue and emerged from it a better programmer. Dealing with bugs and errors is what builds our confidence in the long run and teaches us valuable lessons along the way. So, in this talk, we'll discuss different ways of handling errors and making our lives a little better. We'll talk about how code written with effective exception handling strategies can help us to catch bugs early in the software developmental cycle.

uk: вхід без авторизації

name: Handling Errors the Graceful Way in Python


uk: Комп'ютерні науки

description: This talk will have special meaning for Final Fantasy fans, but anyone with nerdy non-coding hobbies should be able to enjoy it. The intended outcome is for attendees to see their own hobbies in a new light, where they can find their own abstract lessons. MMO gaming is becoming a more mainstream hobby, and as a lifelong gamer, I have plenty of experiences and stories to share on how the genre helped my transformation from a shy bundle of nerves to a confident professional. I also have experience applying my problem solving skills from software development into solutions for the community to combat problems that arose within the game.

uk: вхід без авторизації

name: How a popular MMORPG made me a better developer


uk: Комп'ютерні науки

description: [Wicklow Hall 1 on 2022-07-14] MASQUE (https://tools.ietf.org/id/draft-schinazi-masque-01.html) (Multiplexed Application Substrate over QUIC Encryption) is a draft of a new protocol that allows running proxy or VPN services indistinguishable from HTTPS servers. Akamai built a managed proxy service based on the MASQUE protocol to provide egress proxy (https://www.akamai.com/blog/cloud/powering-and-protecting-online-privacy-icloud-private-relay) for iCloud Private Relay. While working on the proxy at Akamai, I wrote a Python client for testing the proxy service. The MASQUE protocol can tunnel traffic through HTTP/3 or HTTP/2, but common Python libraries only support HTTP/1.1. The tunneled traffic can use any protocol on top of TCP or UDP, including all HTTP versions, so MASQUE can be proxied through MASQUE for onion routing. In this talk, I will show that the MASQUE proxy design is simple and yet client implementations are complex. To put everything into context, I will recap how HTTP proxies operate and how HTTP versions differ. I will highlight lessons learned from designing a low-level HTTP client using Python asyncio.

uk: вхід без авторизації

name: How I wrote a Python client for HTTP/3 proxies


How we are making Python 3.11 faster

uk: Комп'ютерні науки

description: [The Auditorium on 2022-07-14] The "Faster CPython" project aims to speed up Python, specifically CPython, by a large factor over the next few releases. The first release to see the benefits of this work is Python 3.11. Python 3.11 includes the following major changes: * Adaptive specializing interpreter (PEP 659) * Consecutively allocated execution frames * Zero cost try-except * More regular object layout * Lazily created object dictionaries. I will describe each of these, describing how each helps speed up Python, and how they interact with each other. I will end the talk with some possible directions for Python 3.12.

uk: вхід без авторизації


HPy: a better C API for Python

uk: Комп'ютерні науки

description: [Liffey A on 2022-07-13] The official Python C API is specific to the current implementation of CPython. It has served us well and forms the basis upon which our entire extension ecosystem rests. However, it exposes a lot of internal details which makes it hard to implement it for other Python implementations (e.g. PyPy, GraalPython, Jython, IronPython, etc.), and prevents major evolutions of CPython itself, such as using a GC instead of refcounting, or removing the GIL. This is where HPy comes in. It's a new C API designed from the ground up according to the following goals: * running much faster on alternate implementations, and at native speed on CPython * making it possible to compile a single binary which runs unmodified on all supported Python implementations and versions * being simpler and more manageable than the Python/C API * providing an improved debugging experience. We'll discuss its current status and show how existing extensions can be gradually ported to it.

uk: вхід без авторизації


I have to Confess, I still Love Pandas

uk: Комп'ютерні науки

description: [Liffey Hall 1 on 2022-07-13] In this talk, Cheuk will reapproach Pandas as someone who is a fluent user of the library. Cheuk will review why Pandas is useful to work with data and what advantage it has over the alternatives methods. As Pandas has many functionalities, Cheuk will dissect the discussion on the potential usage of Pandas in day-to-day data science workflow: data inspection, data cleaning, feature engineering etc. After discussing the advantage of using Pandas, Cheuk will discuss the flip side. What makes Pandas a difficult tool to use at first. Here Cheuk will share how she used it efficiently with new users and some fundamental concepts about Pandas. This talk is for Pandas users (new and old) or potential users. Those who are familiar with Pandas may get a refreshing idea from a new angle about the tool. And for the new users, it will be a good startup guide to make the journey of using it a bit easier.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: [Liffey Hall 2 on 2022-07-14] This work is an example of an intersection of a non scientific field with computer science and mathematics, trying to quantify, measure and identify non mathematical phenomena in the language of mathematics. It is important because it could be the basis of the scientific approaches that the next generation policy makers, voters, non profit social organizations and governments could use to make life changing decisions for their citizens. 2 The questions that this study tries to answer is whether a neural network can learn biases from the news media based on perceived bias scores obtained from independent agencies. It also seeks to answer whether any of these political leanings of the news media affect the vulnerability of their consumer when it comes to fake news. The results of this experiment aim to show Conclusions: 1. SVMs perform better clustering with respect to the categories than neural networks, however the maximum does not cross 67% 2. The most significant conclusion from this work is that though there is a perceived bias when it comes to news agencies, when looked at from a neural networks standpoint, it is negligible. Mainstream news agencies are not able to polarize a neural network with inherent biases in their headlines. 3. There may be topical biases that need to be examined by using an Entity linking and bias calculation approach 4. Most mainstream news agencies do not make the consumer vulnerable to believing fake news. This study needs to be conducted with data from popular social media ”news” groups or popular TV shows that masquerade as news but may technically not even be news channels. It is safe to conclude that the perceived bias that stems from social media polarization is being extended to news media when their contribution to the polarization may be negligible.

uk: вхід без авторизації

name: Is the news media polarized? Or are we conditioned to think it is?


Jupyter - Under the Hood -

uk: Комп'ютерні науки

description: [Liffey Hall 2 on 2022-07-15] Jupyter Notebooks at their core are just JSON documents that contain all your code, markdown styles and outputs. Yet when you run a notebook, there's a lot that's happening under the hood - from starting a session with the notebook server, to launching an IPython kernel, and a rich Web UI communicating with the notebook server and the IPython kernel using Jupyter's REST APIs and ZMQ websockets. We will explore the Jupyter ecosystem and see how this system comes together. The architecture of all the offerings in the Jupyter Project (such as the classic Jupyter Notebook), the newer JupyterLab IDE, or the scalable multi-user environment - JupyterHub is completely distributed. At their core, there's a front end client like a web browser or a qt console that talks to the Notebook server using its many APIs (like the kernel API) and to the language kernel (in our case IPython) using ZMQ Sockets, allowing the Jupyter architecture to scale easily. In this presentation, we look closely at these REST API calls, and the ZMQ socket traffic using simple tools like the browser's network tab. We will also try to manipulate a notebook using simple code to get a full appreciation of these internals.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: [The Auditorium on 2022-07-13] If real life was a superhero movie, we’d have all the ingredients needed for a hero’s rescue. So many “AI” algorithms are being applied to EU education, employment, and public safety systems that you might wonder if the TV series “Black Mirror” is fiction or a blueprint for nefarious actors. Luckily, there are codes to keep dystopia at bay, whether from the fictional Justice League or from real-life courts of justice. This talk discusses both, and is aimed at software engineers, architects, designers, testers and product/project managers who want to slow the Automation of Everything, but don’t know how.

uk: вхід без авторизації

name: Keynote: Dodging AI Dystopia: you can't save the world alone


uk: Комп'ютерні науки

description: [The Auditorium on 2022-07-14] Killer robots may sound like something from a movie, but in recent years weapons have been developed that can select targets and attack without any human input, and expert systems have been used to assist in military targeting. Some argue that this is a positive development, because automation can increase precision in targeting and reduce civilian casualties. However, others point out that highly automated systems do not have a good track record in complex and high-stakes real-world situations, and military conflict is unlikely to be better. This talk will outline the technological underpinnings of autonomous weapons and automated targeting systems, as well as examining the legal and ethical debate over these systems that has been happening at the UN over the past decade.

uk: вхід без авторизації

name: Keynote: Killer Robots Considered Harmful


uk: Комп'ютерні науки

description: [The Auditorium on 2022-07-15] CPython’s “Global Interpreter Lock”, or “GIL”, prevents multiple threads from executing Python code in parallel. The GIL was added to Python in 1992 together with the original support for threads in order to protect access to the interpreter’s shared state. Python supports a number of ways to enable parallelism within the constraints of the GIL, but they come with significant limitations. Imagine if you could avoid the startup time of joblib workers, the multiprocess instability of PyTorch’s DataLoaders, and the overhead of pickling data for inter-process communication. The “nogil” project aims to remove the GIL from CPython to make multithreaded Python programs more efficient, while maintaining backward compatibility and single-threaded performance. It exists as a fork, but the eventual goal is to contribute these changes upstream. This talk will cover the changes to Python to let it run efficiently without the GIL and what these changes mean for Python programmers and extension authors.

uk: вхід без авторизації

name: Keynote: Multithreaded Python without the GIL


Leading & growing software teams

uk: Комп'ютерні науки

description: [Wicklow Hall 1 on 2022-07-15] Software development is a team game. As you progress through your career, you might end up in a leadership role, taking care of your own team, or even of multiple teams. As a team lead, it’s up to you to establish a good working rhythm, set the right expectations, communicate up and down the chain of command and effectively help your team grow in both technical and non-technical terms. As a team lead, you want to enable your team to reach its full potential. The main goal of this talk is to provide pragmatic real-life examples, about how to achieve those things. We are going to cover the following topics: 1. What’s the role of a team lead? 2. Managing expectations & responsibilities. 3. Establishing a good work rhythm. 4. Establishing a good form of communication. 5. What does team growth look like? This talk is the natural sequel of the following talks from previous EuroPythons: - EuroPython 2017 - Practical Debugging - Tips, Tricks and Ways to think - https://www.youtube.com/watch?v=9Ys4gCUtTh8 - EuroPython 2018 - Django structure for scale and longevity - https://www.youtube.com/watch?v=yG3ZdxBb1oo - EuroPython 2019 - Software patterns for productive teams - https://www.youtube.com/watch?v=fEy68VRmOeQ"

uk: вхід без авторизації


uk: Комп'ютерні науки

description: EuroPython 2022 - Lessons learnt from building my own library - presented by Stephanos [Liffey B on 2022-07-14] This talk is about all the mistakes that I made while building a library, how I would have avoided making them if I started today, what turns I would have taken differently, what choices I made, and why I made them. Examples are drawn from the building of a specific library, but it's not about a specific one. The aim is to give some insights more into how to make choices when building a library available to the world, and less on the specific set of choices that I made. It will not be technically challenging, but some familiarity with the Python ecosystem is advised." This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-nc-sa/4.0/

uk: вхід без авторизації


Let's talk about JWT

uk: Комп'ютерні науки

description: [Wicklow Hall 1 on 2022-07-14] JSON Web tokens dominated the way we give access to APIs and how we carry data from users, but to use JWTs safely we need to understand how they came to life and how JWTs can be useful. In this talk we will take a closer look at the famous three-part structure that forms a JSON Web Token, and the claims each JWT can carry. But knowing it’s history and structure is not enough, we need also to understand the algorithms used in creating a token and how you can use JWTs as access tokens or as ID tokens. After understanding JWTs on a deeper level, we will create and validate a JWT together using the PyJWT library and discuss things you should avoid doing to be safer when using JWTs in your projects. 1. How did JWT come to life? Talk about the JOSE specification; 2. What actually is a JSON Web Token and its structure: header, payload, and signature; 3. What is a claim and its standardization efforts; 4. The different types of algorithms that can be used to create JWTs and what is JWKs; 5. Let's create a token together using PyJWT; 6. What is an access token and an ID token; 7. Things to avoid to be safer with JWTs

uk: вхід без авторизації


Lint All the Things!

uk: Комп'ютерні науки

description: [Liffey B on 2022-07-14] Many teams document the conventions for their projects. However, documentation gets out of date, forgotten, or sometimes ignored. Simple documentation requires team members to constantly remember all the ‘rules’ for your project. You can better enforce those rules and free up your team members to think about harder problems using linting tools like flake8, import linter, and pre-commit. These tools provide tons of useful stuff out of the box, but you can push them so much further with customization. This allows your project to formally document conventions, but also enforce them automatically on every commit, merge, and build. This can make code reviews faster and more focused on the problems your code is meant to solve. This talk will introduce tools like flake8, import linter, and pre-commit along with some of their built-in functionality. Then, we’ll briefly explore some ways to customize them to fit your projects’ specific needs. Some examples of custom linter rules we’ll tour are: - Code formatted automatically and uniformly - Code doesn’t import across architecture layers violating separation of concerns - Common conventions are used - Common anti-patterns are avoided - Specific layers are fully tested - Proper git commit message formatting - Merge commits don’t exist in topic/feature branches Finally, we’ll discuss ways to use those custom linter rules on every commit, merge, and build with continuous integration or git hooks. By the end of the talk, you’ll see several real-world linter rules used on Kraken, which is a large Django-based project used to supply green energy to millions of users across the world. In addition, expect no shortage of ideas for your own projects along the way!"

uk: вхід без авторизації


uk: Комп'ютерні науки

description: With the staggering dominance of public cloud providers, dev teams across the globe are increasingly focusing time and energy on optimizing their cloud development and deployment flows. The traditional deploy-and-test cycles against public clouds can become slow and tedious, where developers are often facing several minutes of idle times between deployments that need to be frequently triggered during testing & debugging. In this session, we provide a hands-on introduction to LocalStack (39k+ Github stars), a fully functional local AWS cloud stack. With LocalStack, applications can be developed entirely on your local machine, reducing dev&test cycles from minutes to seconds. The session covers interactive live coding to showcase common scenarios and use cases, different settings for local debugging of Lambdas and containerized apps (e.g., ECS/EKS), as well as some advanced new features that can radically improve productivity and team collaboration patterns. We will also glance over the large ecosystem of tools that LocalStack natively integrates with - from IaC frameworks like Terraform or Pulumi, to application frameworks like Serverless or Architect, to a whole suite of tools provided by AWS itself (CDK, SAM, Copilot, Chalice, etc). We'll wrap up the session with a deep dive into some of the Python internals of LocalStack, which reveals some interesting architectural patterns and hidden gems!

uk: вхід без авторизації

name: LocalStack: Turbocharging dev loops and team collaboration for cloud applications


uk: Комп'ютерні науки

description: [Liffey Hall 1 on 2022-07-15] Task of Machine Translation engine evaluation may be very challenging. Quality of Machine Translation varies greatly depending on domain and language pair. Different MT engines may have different interfaces or APIs and different requirements to run. To add to that, even definition of a good translation may be debatable, with any automatic MT quality metric providing only approximation of actual translation quality. That's why having universal evaluation framework for this task is very important. In our work we tried to create such framework. 1) We defined base translation class that unified all file handling, batch creation and result processing. As a result of that, only work needed to support new MT engine was creation of small child class that implemented couple of simple functions. That allows us to easily extend our framework to MT engines and new language pairs. 2) We defined set of test datasets and provided a way to add new datasets to this set. For our evaluation our aim was to create test data that covers both general and healthcare domains EMEA dataset (https://opus.nlpl.eu/EMEA.php), OPUS-100 (https://opus.nlpl.eu/opus-100.php), Paracrawl (https://paracrawl.eu/) and several others. But our data preparations scripts can be easily extended to other domains and datasets as well. 3) We defined a set of quality metrics to evaluate results of MT engines. Metrics that we used included BLEU (https://github.com/mjpost/sacrebleu), BERTScore (https://github.com/Tiiiger/bert_score), ROUGE (https://github.com/pltrdy/rouge), TER and CHRF (both also from sacrebleu implementation). Beside MT evaluation framework we will present our own evaluation results. For our evaluation we used cloud based engines - Azure Translator (https://azure.microsoft.com/en-us/services/cognitive-services/translator/), Google Translate (https://cloud.google.com/translate/), as well as open-source engines - Marian MT (https://huggingface.co/transformers/model_doc/marian.html), NVIDIA's NeMo (https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/nlp/machine_translation.html), Facebook's MBart 50 (https://huggingface.co/facebook/mbart-large-50-one-to-many-mmt), Facebook's M2M100 (https://huggingface.co/facebook/m2m100_418M). For open source engines we tried to use Huggingface's transformer implementation whenever possible. But as we mentioned our framework was designed in a way to be easily extendable to other MT engines and underlying frameworks. We also will present evaluation results for NeMo and MarianMT engines that we fine-tuned specifically for healthcare domain. While these particular results may rather specific to our use case, they help to highlight how our framework can be extended to custom MT engines as well.

uk: вхід без авторизації

name: Machine Translation engines evaluation framework


Making AI Happen at Your Company

uk: Комп'ютерні науки

description: [Liffey Hall 1 on 2022-07-13] Many incumbents are transitioning to new technologies while their businesses operate on systems that are years or decades old. Introducing new technologies is not just about introducing Open Source or introducing community culture or working agile or SCRUM or explaining the complicated technology stuff to executives. The truth is: it requires all of it and likely even more. Mastering innovation requires having many balls in the air at once. In this talk I’ll present a transformation use case of an established player including our best practices and anti-patterns. We will discuss the following aspects: * From idea to strategy * Assessing the status quo * Introducing Python and Open Source and what to use (or not) * Legacy is in the the house, still * Getting all departments on the same page * Introducing a community-driven collaborative culture

uk: вхід без авторизації


uk: Комп'ютерні науки

description: [The Auditorium on 2022-07-13] Python 3.10 has been recently released and among many exciting new features, one of the biggest improvements is the inclusion of a whole new set of changes focused on improving the error messages across the interpreter and the general user experience when dealing with error messages. The new error messages have been one of the most welcomed features from very different sets of users ranging from Python teachers and educators, first-time learners, industry professionals and data scientists. In this talk, we will cover: * What are the new improvements featured in Python 3.10. * Exciting new changes and improvements that will feature in Python 3.11. * How these improvements are useful to different sets of users from people learning Python to experienced programmers. * How the new PEG parser has unlocked adding new custom syntax errors. * How these improvements were implemented and what challenges the CPython core team faced to get them working reliably. * How users can contribute to adding new error messages: what is the workflow, how the errors are reviewed by the core team and where to find resources and help. No matter who you are and what you do with Python, there is an improvement that will probably make you smile.

uk: вхід без авторизації

name: Making Python better one error message at a time


uk: Комп'ютерні науки

description: [Liffey Hall 1 on 2022-07-14] Data science experiments have a lot of moving parts. Datasets, models, hyperparameters all have multiple knobs and dials. This means that keeping track of the exact parameter values can be tedious or error prone. Thankfully you're not the only ones facing this problem and solutions are becoming available. One of them is Hydra from Meta AI Research. Hydra is an open-source application framework, which helps you handle complex configurations in an easy and elegant way. Experiments written with Hydra are traceable and reproducible with minimal boilerplate code. In my talk I will go over the main features of Hydra and the OmegaConf configuration system it is based on. I will show examples of elegant code written with Hydra and talk about ways to integrate it with other open-source tools such as MLFlow.

uk: вхід без авторизації

name: Managing complex data science experiment configurations with Hydra


uk: Комп'ютерні науки

description: [Wicklow Hall 1 on 2022-07-13] As developers we often have to deal with legacy projects and, at the same time, we want to keep the quality and security of our deliverables under control. As soon as we start running some linter (like Pylint or Flake8) on such a legacy project, there is a huge number of violations. To handle those issues, we might want to start by only looking at the changed files in a pull request instead of the entire project, for example by using _git diff_ _pylint `git diff --name-only --diff-filter=d`_ During this talk I’d like to push this concept a bit further and outline an approach and philosophy that can be helpful in dealing with code quality : Clean as you code. 1. What is "Clean as you code"? - Not only about violations: It can be extended to code coverage and all code metrics in general. - The quality you want to measure should be based only on recent changes. 2. "Clean as you code" matters? - It helps your team stay focused on delivering new features - It helps you deal with technical debt incrementally: Sometimes you might need to modify old code, and, at that point, you might be able to fix existing violations 3. How to apply "Clean as you code"? - Shaping a *quality gate* in order to define code quality standards for the software delivered by your team today - Using appropriate tools (like SonarQube)

uk: вхід без авторизації

name: Managing the code quality of your project. Leave the past behind: Focus on new code


Maps with Django

uk: Комп'ютерні науки

description: [Wicklow Hall 1 on 2022-07-14] A _map_ in a website is the best way to make geographic data easily accessible to users because it represents, in a simple way, the information relating to a specific geographical area and is in fact used by many online services. Implementing a web _map_ can be complex and many adopt the strategy of using external services, but in most cases this strategy turns out to be a major data and cost management problem. In this talk we'll see how to create a web _map_ with the *Python* based web framework *Django* using its *GeoDjango* module, storing geographic data in your local database on which to run geospatial queries. Through this intervention you can learn how to add a _map_ on your website, starting from a simple _map_ based on Spatialite/SQLite up to a more complex and interactive _map_ based on PostGIS/PostgreSQL.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: In recent years, we have seen many improvements in Python Garbage Collection but there are some instances when it doesn’t work as expected. This results in memory crunch for the application leading it to crash. Although there are multiple ways to overcome the memory challenges, sometimes it is difficult to find what we can improve in our code and infrastructure that can make them memory efficient. In such cases, it helps to have an understanding of what is going on behind the curtains at a low level where memory is being managed. This presentation aims to give a quick overview of 1. How CPython manages the Memory allocation 2. Common memory errors we see in day to day production code and how we can improve them We will share what we have learned so far and encourage you to try it with your own projects. We'll walk through a simple example, with screenshots and code wherever required."

uk: вхід без авторизації

name: Memory Problems, Did Collector Forgot to Clean the Garbage?


uk: Комп'ютерні науки

description: The Mercury is a new framework for converting Jupyter Notebook to a web app. The widgets are constructed based on the YAML config (similar to R Markdown). The user can tweak the values of the widgets and execute the notebook from the top to the bottom. The notebook sharing is as simple as sending a link. What is more, you can decide whether to show or hide the code. In the talk, I would like to show how the Python notebook can be converted to a web app and deployed to the cloud.

uk: вхід без авторизації

name: Mercury - Build & Share Data Apps from Jupyter Notebook


uk: Комп'ютерні науки

description: EuroPython 2022 - My journey using Docker as a development tool - presented by Haseeb Majid [Wicklow Hall 1 on 2022-07-13] Docker is a prevalent tool in our industry today, it is widely used for several proposes. In this talk, I would like to describe the journey I have taken with Docker and how I have learnt to use it with Python for local development. The talk will mostly focus on how we can use Docker to run all of our development tasks, so we hardly need to have anything installed locally. To the point where you can work on almost any project with nothing more than an editor, your terminal and Docker. We will look at how we can take a simple FastAPI web service and dockerise it. The talk will be a step by step guide, starting of with a simple Docker image and improving it iteratively. - Firstly use a simple Docker image - Then add docker-compose to dockerise its dependencies such as database - Then see how we can run tests (and other tasks) directly in our Docker container - How to use this with Github Action - Then look at multistage builds - Then split this into development and production image - How can we handle git dependencies that require an SSH key to be injected into the Docker image at build time I’d be a little more specific. Like, how we handle private pip packages in our docker images. This talk is for you if you are a casual user of Docker and know a few of its features but want to improve your Docker skills. This is NOT a talk for you if you are an expert in Docker and know most of its features. Code: https://gitlab.com/haseeb-slides/docker-as-a-dev-tool Slides: https://docker-as-a-dev-tool.haseebmajid.dev This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-nc-sa/4.0/

uk: вхід без авторизації

name: My journey using Docker as a development tool: From Zero to Hero


uk: Комп'ютерні науки

description: Distributing Python GUI applications to end users is a challenge: will they need to install Python? If so, which version? If not, how do they install the application? From a random ZIP file? How native does the process feel? Will their system trust your code? For a fluid experience, it needs to be signed and (on macOS) notarized beforehand. Welcome to `pup`, the tool that the Mu Editor development team has created to package and distribute it in platform-native formats to Windows and macOS users around the world. In this session I will show how `pup` can be used to package GUI Applications for distribution: natively on Windows and macOS, and in early stages of development for distribution-agnostic Linux artifacts. In short, if it's `pip`-installable it is `pup`-packageable! I will then describe the way `pup` works (and how it differs from comparable tools) leading on to a call-for-action moment, where I'll share its current state of development, what's good, what's bad, and where I'd like it to be headed to. I'll wrap up the talk with a set of future-looking thoughts that `pup` has helped identify not only on the specifics of CPython's distribution, but also on the Python ecosystem as whole.

uk: вхід без авторизації

name: Native Packaging of GUI Apps on Windows and macOS


uk: Комп'ютерні науки

description: The method consists of three main steps: First, network embedding formulation performed on each user specific behavior network; Then, embeddings weight distribution estimated through intermediate layers of network with final layer for target (item purchased as labels); Finally, both factors: (a) Learned weights from implicit data (cross-domain) and (b) explicit factors from domain data used by multi-factorization method for recommendations. The proposed method transfers knowledge across implicit and explicit factors and associated dimensions. The suggested approach tested real-world data with evidence of outperforming existing algorithms with significant lift in recommendation accuracy. Empirical experimentation outcomes illustrate the potential of both factors for making effective recommendations.

uk: вхід без авторизації

name: Network Embeddings based Recommendation Model with multi-factor consideration


uk: Комп'ютерні науки

description: Since its inception, online voting has been an appealing but controversial technology. Indeed, what seems like a modern way of making vote cheaper and more convenient is often considered by activists and researchers as a pandora's box unleashing never-ending privacy and authenticity concerns. However with Covid 19 shrinking our public interaction, many have considered the benefits to overcome the theoretical issues and the online voting systems have skyrocketed like never before... The Neovote voting system has been massively used in France: tenths of University, hundreds of private companies, and, more importantly, it was chosen to organize 3 of the 5 main primary elections for the French Presidential election of 2022 (Primaires de l'Écologie, Les Républicains and Primaire Populaire). Neovote claims to have the highest possible level of security, the voter being even able to access the final ballot box to do the recount by himself and ensure his own vote has been taken into account! So challenge accepted, this talk will walk you through the Neovote voting system to understand why their claims are "slightly" exaggerated.

uk: вхід без авторизації

name: Online voting system used for primary elections for the French Presidential, must be secure right?


uk: Комп'ютерні науки

description: Here, I elaborate on why we should develop tools that will allow us to build pre-trained models in the same way that we build open-source software. Specifically, models should be developed by a large community of stakeholders who continually update and improve them. Realizing this goal will require porting many ideas from open-source software development to building and training models, which motivates many threads of interesting research and opens up machine learning research for much larger participation.

uk: вхід без авторизації

name: Open Science: Building Models LIke We Build Open-Source Software


Packaging Python in 2022

uk: Комп'ютерні науки

description: The Python packaging landscape is experiencing a renaissance but along with new standards and new tools comes a lot of new choices when publishing. setup.cfg or pyproject.toml? Do you need a setup.py instead or in addition? There can be a lot of confusion but understanding modern trends can make sharing your code easier than ever before.

uk: вхід без авторизації


Packaging security with Nix

uk: Комп'ютерні науки

description: Managing securely dependencies is becoming an increasing concern of the industry. Here, we showcase how Nix, a functional-oriented package manager, can get us very far and close class of vulnerabilities that PyPI / pip had in the past, e.g. rogue PyPI packages that steals personal data.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: Dealing with extreme heatwaves can be challenging, it has become the necessity to understand the land surface temperature (LST) change and its driving factors to reduce the impact and achieve more sustainable planning methods for city growth. This module will help you understand how to calculate LST from the openly available satellite imageries and merge it with urban morphological factors (like building height, building count, FSI, building block coverage, etc.) to predict the temperature trend and mitigate the impact. We will demonstrate an end-to-end methodology using geospatial Python libraries to understand the use of spatial regression methods taking into account the variation over time. This talk will also throw light upon: - Getting the large imagery datasets into DL friendly format - Spatial aggregation of different variables - Understanding correlation between variables for feature engineering - Application & comparison of different regression methods on the same data - Future scope We'll also showcase the geo-visualization portal we created and the technologies used, how you can use Python to convert large GeoJSON output to light vector tiles, and create a seamless experience for the user through an intuitive front-end.

uk: вхід без авторизації

name: Predicting urban heat islands in Calgary


uk: Комп'ютерні науки

description: Python introduces Protocols to support static duck typing, where static type checkers (mypy) and other tools can verify code correctness prior to runtime. This was added in order to circumvent explicitly inheriting from ABCs (Abstract base classes) which is ""unpythonic and unlike what one would normally do in idiomatic dynamically typed Python code"" - according to PEP 544. We will explore the different use cases for Protocols and how to use them correctly.

uk: вхід без авторизації

name: Protocols - Static duck typing for decoupled code


Protocols in Python: Why You Need Them

uk: Комп'ютерні науки

description: Protocols have been around since Python 3.8. So what are they, and how can they help you write better code? And how are they different from Abstract Base Classes? In this talk I will introduce you to both concepts (ABCs and Protocols), and show you by example how they can make your life easier, and your code cleaner.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: In this talk we will introduce PyArrow and talk bout the transformation that the Arrow format is allowing in the Data Analytics world. PyArrow provides an in-memory format, a disk format, a network exchange protocol, a dataframe library and a query engine all integrated in a single library. But the Arrow ecosystem doesn't stop there and allows you to work integrating multiple different technologies. It can be a swiss army knife for data engineers and it integrates zero cost with NumPy and Pandas in many cases."

uk: вхід без авторизації

name: PyArrow and the future of data analytics


uk: Комп'ютерні науки

description: I had an idea for a debugging solution for Python that doesn't require complicated configuration like PyCharm. I released PySnooper as a cute little open-source project that does that, and to my surprise, it became a huge hit overnight, hitting the top of Hacker News, r/python and GitHub trending. In this talk I'll go into: * How PySnooper can help you debug your code. * How you can write your own debugging / code intelligence tools. * How to make your open-source project go viral. * How to use PuDB, another debugging solution, to find bugs in your code."

uk: вхід без авторизації

name: PySnooper: Never use print for debugging again


uk: Комп'ютерні науки

description: Visual Studio Code along with GitHub, Codespaces, and Azure Machine Learning have been investing substantially into tools and platforms to make the lives of Python data scientists easier, and we want to share why VS Code is now the #1 tool for Python Data Scientists according to the 2021 Python Software Foundation Developer Survey, and how you can leverage VS Code to take your data science productivity to the next level. This talk will walk through several common Python data science scenarios, showcasing all the productive tooling VS Code has to offer along the way. As a sneak peek, we will be demoing a best in class Jupyter Notebooks experience with VS Code Notebooks, a revolutionary new data cleaning / preparation experience with Data Wrangler in VS Code, collaboration features with GitHub and Codespaces, Azure Machine Learning for deployment, and more!

uk: вхід без авторизації

name: Python & Visual Studio Code - Revolutionizing the way you do data science


uk: Комп'ютерні науки

description: The various areas within humanities and social sciences such as political science, sociology, psychology, economics etc. have evolved to a point where they have been complementing existing qualitative and quantitative methods with methods rooted in data science. This shift in paradigm is primarily driven by real-world, publicly available data sets that cover a variety of scholarly domains and have the potential to solve fundamental research questions in these intriguing fields. There is however a huge bottleneck to be overcome before realizing the full potential of data science in arts, humanities and social sciences; and that bottleneck relates to a fear of programming in students/researchers within these disciplines. Our talk presents some tips and tricks from course modules being taught in Technological University Dublin; the fundamental idea is to present an overview of data science toolkit and how it relates to problem solving in the real world. First we will present ways to make the data science lifecycle being made easy via tools such as Google colab and Jupyter notebooks followed by explaining how showing students the big picture and the workflow lifecycle of a data science technique helps grasps concepts in a very effective manner. We will present examples from exploratory data analysis and classification using data-driven research questions; and look into elegant solutions that can easily be plugged into a social scientist’s skillset. Examples of continuous assessment projects based on different Python libraries are also presented with a view to further establish use of Python as a valuable tool for arts, humanities, and social sciences. Finally, we will give an overview of an Irish research council funded project emanating from a combination of STEM and HUMANIITIES disciplines that aims to perform an economic assessment of anti-immigrant sentiment in Ireland. Various phases of the project will be explained emphasizing particularly those where Python tools play a major role in interpretion of research outcomes. These research outcomes obtained through smart use of Python data analytic tools play a key role in building relationships between data scientists and policy makers in both government and not-for-profit sector.

uk: вхід без авторизації

name: Python for Arts, Humanities and Social Sciences


uk: Комп'ютерні науки

description: Empowering anyone to issue a new release by submitting a pull request? At first glance, it seems like an outlandish idea. Upon further inspection, however, the benefits become clear. One of the biggest sources of friction in software development is packaging and publishing new releases. Somebody has to sift through the commits and write a change-log, tag it with a new version number, and publish the package to PyPI. And usually only one or two people have the access necessary for this last step. The unfortunate result is an infrequent release cadence. Bug fixes and new features are sitting there in the `main` branch, but hardly anybody is benefiting from them because they aren’t in a shipped release yet. Thankfully, there’s a better way: a continuous release process where new versions are auto-published via pull requests — without any manual monkeying. In this environment, all pull requests must include a release file that describes the changes within. This file must also include whether the new version should be a major, minor, or patch release. The continuous integration (CI) process looks for this file, and if one hasn’t been included, the tests will fail, and the pull request won’t be merged. Otherwise, a maintainer merges the pull request, and the CI process then uses the release file to put the description into the change-log, increment the version number, commit, tag, build, and publish the new release to PyPI. So with almost no human input, every code contribution results in a new release in a matter of minutes. Every feature and bug-fix has its own release, without anyone having to remember to package and publish a new version. Plus, if a bug is found, it’s now much easier to trace it to a specific release version. Perhaps the best part is that all contributors get to issue their own releases. What better way to welcome new contributors than to reward them with a dedicated release composed entirely of their own work? Attendees of this talk will take home the following knowledge and skills: * what kinds of problems are introduced by an infrequent release cadence * how automated releases solve those problems and provide myriad ancillary benefits * how to format and parse the release file * how to use continuous integration (CI) systems to automate releases * how to add deployment code so new versions are released when PRs are merged * how to use AutoPub to facilitate this process

uk: вхід без авторизації

name: Python Packaging Automation: Auto-Publish to PyPI via Pull Requests


uk: Комп'ютерні науки

description: New to python 3.11, Exception Groups help you raise and handle errors more robustly than ever before - you will delve deep into the current gaps in python's exception handling mechanisms, and get to know Error Groups, and a new python keyword except*, that can be used to overcome those issues and to write cleaner code.

uk: вхід без авторизації

name: Raise better errors with Exception Groups


uk: Комп'ютерні науки

description: BBC News Labs is an innovation team within BBC R&D, working with journalists and production teams to build prototypes to demonstrate and trial new ideas for ways to help journalists or bring new experiences to audiences. We work in short project cycles to research and build prototypes. We have worked with the BBC's flagship radio news programme production team to enrich programme timelines with metadata to provide enhanced experiences to the audience. We are currently working with local radio teams around the UK to provide the means to capture highlights in live radio for re-use and for social media, reducing the workload for producers, and getting more mileage from linear broadcast programmes. Working in short cycles, it's important for us to be able to quickly build processing pipelines connected to BBC services, test and iterate on ideas and demonstrate working prototypes. We make use of modern cloud technologies to accelerate delivery and reduce friction. In this talk I will share our ways of working, our ideation and research methods, and the tools we use to be able to build, deploy and iterate quickly, the BBC's cloud deployment platform, and our use of serverless AWS services such as Lambda, Step Functions and Serverless Postgres.

uk: вхід без авторизації

name: Rapid prototyping in BBC News with Python and AWS


uk: Комп'ютерні науки

description: I am the author of `streamlit-webrtc` and a member of the Streamlit Creators program (https://streamlit.io/creators) (selected community members). The repository of `streamlit-webrtc` is here: https://github.com/whitphx/streamlit-webrtc My lightning talk about `streamlit-webrtc` at PyCon JP 2021 is available: https://youtu.be/_LuLs8H1gJc Articles about this library: * Developing Web-Based Real-Time Video/Audio Processing Apps Quickly with Streamlit: https://towardsdatascience.com/developing-web-based-real-time-video-audio-processing-apps-quickly-with-streamlit-7c7bcd0bc5a8 * Real-Time Video Streams With Streamlit-WebRTC: https://betterprogramming.pub/real-time-video-streams-with-streamlit-webrtc-bd38d15f2ef3 As linked from the repo, demo apps I have developed are available online: * Demo showcase including real-time object detection: https://share.streamlit.io/whitphx/streamlit-webrtc-example/main/app.py * Source code: https://github.com/whitphx/streamlit-webrtc-example/blob/main/app.py * Real-time Speech-to-Text: https://share.streamlit.io/whitphx/streamlit-stt-app/main/app_deepspeech.py * Source code: https://github.com/whitphx/streamlit-stt-app * Real-time style transfer: https://share.streamlit.io/whitphx/style-transfer-web-app/main/app.py * Source code: https://share.streamlit.io/whitphx/style-transfer-web-app/main/app.py * Real-time Tokyo 2020 Pictogram: https://share.streamlit.io/whitphx/tokyo2020-pictogram-using-mediapipe/streamlit-app * Source code: https://github.com/whitphx/Tokyo2020-Pictogram-using-MediaPipe * Video chat: online demo is not available because it does not have an auth mechanism and is only for private use. * Source code: https://github.com/whitphx/streamlit-video-chat-example

uk: вхід без авторизації

name: Real-time browser-ready computer vision apps with Streamlit


Writing Faster Python 3

uk: Комп'ютерні науки

description: EuroPython 2022 - Writing Faster Python 3 - presented by Sebastian Witowski [The Auditorium on 2022-07-14] Did you know that Python preallocates integers from -5 to 257? Reusing them 1000 times, instead of allocating memory for a bigger integer, can save you a couple milliseconds of code’s execution time. If you want to learn more about this kind of optimizations then, … well, probably this presentation is not for you :) Instead of going into such small details, I will talk about more “sane” ideas for writing faster code. After a brief overview of different levels of optimization and how they work in Python, I will show you simple and fast ways of measuring the execution time of your code and finally, discuss examples of how some code structures could be improved. You will see: * The fastest way of removing duplicates from a list * How much faster your code is when you reuse the built-in functions instead of trying to reinvent the wheel * What is faster than the “for loop” * If the lookup is faster in a list or a set * When it’s better to beg for forgiveness than to ask for permission This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-nc-sa/4.0/

uk: вхід без авторизації


Writing secure code in Python

uk: Комп'ютерні науки

description: Is your Python code secure? This talk will show how some inattentions, mistakes and assumptions that we, as developers, carry in our code can lead to serious vulnerabilities in our applications. All of that, of course, with lots of examples! At the end, the talk will present a simple way to audit Python code in order to facilitate the maintenance of your security with the identification of possible vulnerabilities. - Learn how `eval()`, pickle, and pip are vulnerable to arbitrary code execution - Understand the importance of cryptographically-secure randomness - Learn how to audit your code and keep your programs secure - ... and more!"

uk: вхід без авторизації


uk: Комп'ютерні науки

description: Introduction and Motivation: The goal of many within the Python community has been to increase the reach of computer science education in classrooms and institutions around the world. Various studies have pointed out the problem-solving and planning abilities forged by computer science coursework (Salehi et al., 2020; Arfé et al., 2020), and other research has investigated the social stigma (McCartney et al., 2017) and instructor bottlenecks (Raman et al., 2015) that inhibit accessibility to computer science courses. In recent years, significant steps have been taken towards increasing computer science education. However, most of these efforts are largely focused on the introduction of standalone computer science courses rather than integrated coursework. In this talk, we explore the seldom-discussed idea of integrating programming into traditional, non-computing fields such as mathematics, social studies, and science. Interdisciplinary coursework is not a new concept in education: students in an English class are expected to be able to understand, produce, and analyze data visualizations, and mathematics students learn to communicate their work formally in language (Lynch, 2020). However, the current belief in the education realm is that computer science is only for a small sect of highly capable students expressing a deep career interest in the field, a clear misconception that has been debunked by studies (Patitsas et al., 2019) and the experiences of countless professionals applying computational resources in traditionally non-computing industries. As a result of such factors, computer science education is fundamentally isolated from the core curriculum, which is to the detriment of learners and the fields that they pursue. In support of the discussion, we unveil the results of surveys given to physics students enrolled in college-level coursework. In this case-study, students of varied backgrounds in computing are given a lecture on chaotic dynamical systems, and are then shown Python-implemented Runge-Kutta numerical integration methods and simulations to solve and visualize the systems. The insights provided by these students provide great perspective on benefits, reception, and nuances in integrating Python into core classes. Overwhelmingly, many indicated that they believed that the Python demonstrations aided in their understanding of the material. For the attendee, this exclusive look at a case-study will help develop a well-informed perspective on the role of Python beyond just computer science. We conclude the presentation with recommendations for integrated Python coursework, calling for global policymakers and Python community members to step forward and do their part as stakeholders. We pinpoint why Python is the best choice, touching on factors such as its readable syntax, high volume of functional open-source libraries, ever-growing industry demand. The versatility afforded by such elements makes learners of all backgrounds better equipped for solving the challenging problems they will face in their careers. Community members in attendance will gain essential knowledge regarding the issue of Python integration, making them better equipped to argue on behalf of the educational community. For students, seeing how complex differential equations can be numerically approached and solved by computers, understanding how new solar systems are discovered by crunching vast astronomical data, and witnessing a computer rapidly sequence a lengthy DNA chain forges an instant, permanent connection that fosters career interests and prepares students for an increasingly computerized professional world. We seek to advocate for every student to have this opportunity. The key takeaway for talk attendees will be a fresh perspective on the issue of Python’s absence beyond the confines of computer science coursework, as well as methods for mitigation and steps that they can take. Changing education with programming is a long-term investment that will unequivocally accelerate student preparedness and capabilities, but it begins with the focused efforts of an informed public rallying in support of progress. Schedule and Format: 5 minutes: The problem of computer science in education and why we should focus on increasing Python’s role in non-computing classes 10 minutes: Presenting research: a case study on introducing scientific Python to physics students 5 minutes: Why Python is the best option and the role and impact on the Python community 5 minutes: Proposals for Solutions to the Problem 5 minutes: Question and Answer Session"

uk: вхід без авторизації

name: Revolutionizing Education: How Python is Essential Beyond CS


uk: Комп'ютерні науки

description: scikit-learn is an open-source scientific library for machine learning in Python. Since its first release in 2010, the library gained a lot of traction in education, research and the wider society, and has set several standards for API designs in ML software. Nowadays scikit-learn is of one the most used scientific library in the world for data analysis. It provides reference implementations of many methods and algorithms to a userbase of millions. With the renewed interest in machine-learning based methods in the last years, other libraries providing efficient and highly optimised methods (such as for instance LightGBM and XGBoost for Gradient-Boosting-based methods) have emerged. Those libraries have encountered a similar success, and have put performance and computational efficiency as top priorities. In this talk, we will present the recent work carried over by the scikit-learn core-developers team to improve its native performance. This talk will cover elements of the PyData ecosystem and the CPython interpreter with an emphasis on their impact on performances. Computationally expensive patterns will then be covered before presenting the technical choices associated with the new routines implementations, keeping the project requirements in mind. At the end, we will take a quick look at the future work and collaborations on hardware-specialised computational routines.

uk: вхід без авторизації

name: Scaling scikit-learn: introducing new sets of computational routines


uk: Комп'ютерні науки

description: Scalpel is a Python Static Analysis Framework. It provides essential program analysis functions for facilitating the implementation of client applications focusing on statically resolving dedicated problems. We aim to provide Scalpel as a generic Python static analysis framework that includes as many functions as possible (e.g., to easily build inter-function control-flow graph, to interpret the import relationship of different Python modules, etc.) towards facilitating developers to implement their dedicated problem-focused static analyzers. Scalpel’s Github page has received around 100 stars from researchers and practitioners around the world in only three-month after being made public.

uk: вхід без авторизації

name: Scalpel: The Python Static Analysis Framework


uk: Комп'ютерні науки

description: Every phase across the end-to-end machine learning lifecycle exposes a plethora of security risks that often go unnoticed by machine learning practitioners. In this talk we uncover the most critical (and common) security risks in the machine learning lifecycle, covering in-depth concepts as well as practical examples of ways in which these can be exploited as well as resolved and mitigated (analogous to the OWASP Top 10 industry standard). Throughout the talk we will be using a hands on example, where we will be training, packaging and deploying a model from scratch, outlining key risk areas for each step together with tools and practices that can be used to mitigate these risks. By the end of this talk, machine learning practitioners will have a robust intuition of the importance of security best practices throughout the machine learning lifecycle, together with tools and frameworks that can help mitigate undesirable outcomes due to security flaws. - Overview The operation and maintenance of large scale production machine learning systems has uncovered new challenges which have required fundamentally different approaches to that of traditional software. The area of security in MLOps has seen a rise in attention as machine learning infrastructure expands to further critical usecases across industry. In this talk we introduce the conceptual and practical topics around MLSecOps that data science practitioners will be able to adopt or advocate for. We will also provide an intuition on key security challenges that arise in production machine learning systems as well as best practices and frameworks that can be adopted to help mitigate security risks in ML models, ML pipelines and ML services. We will cover a practical example showing how we can secure a machine learning model, and showcasing the security risks and best practices that can be adopted during the feature engineering, model training, model deployment and model monitoring stages of the machine learning lifecycle. - Benefits to the ecosystem This talk will provide practitioners with the intuition and tools to secure production machine learning systems, as well as further the discussion around best practices reinforcing SecOps into MLOps. It will provide best practices on a critical area of machine learning operations which is of paramount importance in production.

uk: вхід без авторизації

name: Secure Python ML: The Major Security Flaws in the ML Lifecycle: (and How to Avoid Them)


Self-explaining APIs

uk: Комп'ютерні науки

description: This talk will present strategies and python tools to create semantically interoperable REST APIs. After the problem statement, various solutions will be presented, including: * contract-first api development with OpenAPI 3, ontologies and controlled vocabularies like the ones published by the European Union (https://op.europa.eu/en/web/eu-vocabularies/authority-tables); * the rdflib and pyld python libraries for processing json-ld and RDF files (https://en.wikipedia.org/wiki/Resource_Description_Framework); * the use of centralized catalogs such as schema.org. Prerequisites: * no prior knowledge of semantics and ontologies; * practical experience with OpenAPI, json schema and data modeling and API design in general.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: When processing data, validating its structure and its type is critical. Bad record types or changes in structure can often result in processing errors or worst in wrong data output. Yet, solving this problem cleanly and efficiently can be challenging. It often results in complicated code logic and increases complexity; consequently decreasing code readability. Pydantic is an efficient and elegant answer to these challenges We expect you'll leave this talk with a good understanding of: - Existing challenges in data validation - What Pydantic Models, Validators, and Convertors are - How to leverage Pydantic in your day to day (using real-life examples) - [Bonnus] How to use Code Generation to create Pydantic Models from any data sources

uk: вхід без авторизації

name: Simple data validation and setting management with Pydantic


Super Search with OpenSearch and Python

uk: Комп'ютерні науки

description: OpenSearch is an open source and free document database with search and aggregation superpowers, based on Elasticsearch. This session covers how to use OpenSearch to perform both simple and advanced searches on semi-structured data such as a product database. Search is pretty useful inside applications, so we'll also discuss how to connect to OpenSearch from existing Python applications, work with data in the database, and perform search and aggregation queries from Python. This talk is recommended for Python developers whose applications are ready to gain some search superpowers.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: We will go through the Why? How? and What? of Model Explainability to build consistent, robust and trustworthy models. We explore the inability of complex models to deliver meaningful insights, cause-effect relationships and inter-connected effects within data and how explainers can empower decision makers with more than just predictions. We evaluate an intuitive game-theory based algorithm, SHAP, with a working implementation in Python. We will also pin-point intersections necessary with domain experts with 2 practical industry applications to facilitate further exploration.

uk: вхід без авторизації

name: Synergize AI and Domain Expertise - Explainability Check with Python: Build trust, transparency and confidence between models and decision makers


Taking charge of your race conditions

uk: Комп'ютерні науки

description: Anybody working with concurrency (threads, processes, or more abstractly workers) eventually encounters a _race condition_. This kind of hair-pulling problem is a staple in computer science. Sometimes you can offload the problem to a lower layer, usually to the database engine, bundling it with atomic transactions. But there are times when you have to take charge of worst-case concurrency scenarios, when your architecture and constraints do not leave room for implicit concurrency management. Then, you need to carefully craft the paths in and out of the critical section, avoiding deadlocks along the way. In this talk, we will focus on how to test your program with a manufactured race condition. If you know it can happen, you'd better cover it in your test suite. We will define a server program called in parallel by multiple clients to illustrate the problem at hand. Then, we will manually set-up a race condition during testing to explore broken behaviour and problematic code. The Python standard library offers all the tools we need to cater for race conditions. We hope the tour we will give allows you to exercise your own race conditions more easily. Why, we even think you will have all the cards in hand to counter them!

uk: вхід без авторизації


Tales of Python Security

uk: Комп'ютерні науки

description: In this session, you'll learn about recent security issues in CPython and the core parts of our ecosystem. You'll hear about the process by which they were filed, how they were reviewed, analysed, shared (when appropriate), resolved and ultimately disclosed to the public. As well as real stories of security vulnerabilities, you'll learn how you can help by responsibly reporting potential issues, and how to protect yourself against common risks, as well as the best ways to find out about major issues and how to respond.

uk: вхід без авторизації


The Design of Everyday APIs

uk: Комп'ютерні науки

description: What makes a good API for a library? Or more importantly, what makes it bad? Implementing an API is an art. It’s the connection between the user and the library itself. How can we optimize that connection to make the experience more pleasing? What makes a user reach for one library over another? What goes into an ergonomic API? This talk will first discuss what makes an API good: documentation, simplicity, consistency, completeness, and flexibility. We will apply those elements by looking at developing and iterating on our own library.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: If you have ever worked on an internal company project, you may feel it deep in your bones. Let’s say that you discovered a need for a generic technological component in your organization’s tech stack. You identified stakeholders, gathered requirements, and started agile iterations on providing it. Then comes a day when you can show the MVP to your internal client! Yet… the client has lost his interest: maybe right now he says that he has already come up with his own temporary solution and he has no intention to switch to another one? Building internal products differs from commercial ones - there is no flow of cash and your clients are fully transparent. In this talk, I would like to share with you my experience and tips connected with developing such internal tools and standards. All of this from the perspective of a member of the Machine Learning Infra team that is delivering its solutions to a rapidly growing ML department in a company whose product is used by 300 million unique users per month. But let’s be specific! I will talk about: - Common pitfalls and try to dig up the reasons for why they happen when developing internal solutions - How one can approach delivering tools (spoiler: pilot programs, guilds, and more!) - Learnings from introducing such approaches (what worked, what didn’t) in our case

uk: вхід без авторизації

name: The intricate art of making your (internal) clients happy: the story from a Python-centered Infra team


uk: Комп'ютерні науки

description: MicroPython is a reimplementation of Python for microcontrollers, originally developed as a result of a Kickstarter campaign. Today, it is an approachable way into programming for many young people, via boards like the Raspberry Pi Pico, the BBC micro:bit and the CodeBug - you can even run it on LEGO bricks! It is increasingly being used in commercial fields as well. MicroPython is helping Python to get into even more places, and making programmers more efficient as it does so. Andy Piper wanted to learn more, so he spent some time travelling and adventuring on the internet, to discover the community and projects around MicroPython. The goal of this session is to briefly explain the What, Why and How of MicroPython. There *will* be circuit boards, and discussion of microcontrollers! We will take a look at examples, from established development boards to brand new ones. We will also acknowledge how MicroPython has been built upon, to enable different ways of working (with CircuitPython), and LEGO robots (via PyBricks). Finally, and most importantly, you'll get a good sense of the places you can find and learn from the MicroPython community, and how you can get involved and contribute!

uk: вхід без авторизації

name: Try Something Different: Explore MicroPython (... tales from an adventurer)


Unfolding the paper windmills

uk: Комп'ютерні науки

description: Research is done on the shoulders of giants. Luckily and unluckily, those giants spoke paper-English and documented their achievements kind of publicly so we could advance the science. In this talk, we will dissect the structure of a paper, looking for the essential points that will help us understand it and implement it. Following we will get our hands dirty and implement the paper using Python. In particular, we will dive into the seminal paper ""Attention is all you need"" and implement a transformer using JAX. The key takeaways from this talk are: - Demystify academic reading. - Understand the Transformer architecture. - An introduction to the JAX ecosystem.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: In this talk we'll cover the following topics: - The problem with the well-known chart taxonomies: starting from "what would you like to show" - Creating a generic chart morphing engine - Advantages of using animation for storytelling - from the presenter's and the audience's perspective - Examples and best practices of using ipyvizzu

uk: вхід без авторизації

name: Use animated charts to present & share your findings with ipyvizzu


uk: Комп'ютерні науки

description: Software has become increasingly complex as it is constructed from a multitude of software components. In many cases the identification of these components are hidden as they are included through implicit dependencies. Without fully understanding the dependencies of your product it is not possible to understand the current vulnerability status of your software product or system. In the past 12 months, there has been an increasing focus on the use Software Bill of Materials (SBOMs) as a key artefact to be delivered with a software product; it will be mandated for all software products in some markets later in 2022. SBOMs which were initially developed to capture the inter-dependencies between components (the focus was on capturing the different types of open source licences used within a product) but with the latest evolution, tracking of vulnerabilities within a product can now be performed. This talk will introduce the SBOM concept and show how Python and its ecosystem can be used to create, manage and use SBOMs as part of your development pipeline.

uk: вхід без авторизації

name: Using Python to manage Software Bill of Materials


uk: Комп'ютерні науки

description: Using Pandas, Python and Plotly to locate potential trend reversals in Stocks, Crypto or any OHLC feed. Learn how to locate Fibonacci retrace levels and predict price reversal zones for the lowest risk entry to a trade.

uk: вхід без авторизації

name: Using python to predict Asset price reversals.


Walk-through of Django internals

uk: Комп'ютерні науки

description: - Talk will start with the introduction of how does end to end web request works internally in Django. - The talk will introduce users to the internals of ORM, database backend, middleware, etc. - The talk will also cover, all processes Django does internally to start a server. - At the end of the talk attendees will be able to understand the internal code structure of Django, how does Django server start, how does Django serves the requests, etc.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: Signing up is on a first-come-first-served basis. The queue is reset every day in the morning. You may talk about / present (almost) everything with certain boundaries: - no promotion for products of companies, no call for 'we are hiring' (but you may name your employer). - conference announcements are limited to one minute only - one LT per person per conference policy: everyone who has not given an LT at this year's conference, yet, is ahead of you in the queue - keep it short, the shorter the presentations, the more people can give a talk

uk: вхід без авторизації

name: EuroPython 2022 - Wednesday's Lightning Talks


What happens when you import a module?

uk: Комп'ютерні науки

description: Modules are a key feature of Python, allowing us to easily reuse our own code and take advantage of publicly available modules from PyPI. It's a rare program that doesn't include at least one "import" statement. But what actually happens when we import a module? How does Python find our file? How does it decide whether it should even try to find our module? And after it finds our module file, how does Python load it into memory, assigning to its attributes? In this talk, I'll walk you through what happens when you "import" a module into Python. The mechanism is surprisingly complex, in no small part because it has to take so many possibilities into consideration. We'll talk about finders and loaders, and about the many ways in which you can customize the module-loading mechanism if you find a need to do so. If you've ever imported a module, then this talk will pull back the curtain a bit, helping you to understand what's happening under the hood.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: I’ve been struggling with gender dysphoria (a debilitating sense of disconnect from the gender assigned to someone at birth) for decades, but it took me until not so long ago to realize what it was, and how it could be treated. Nothing has been the same since. Transitioning and the events leading up to it changed my life, and the experiences I had during my transition changed me as a person, and as a leader. It’s hard for me to open up about this period in my life, not just because it comes with tremendous vulnerability, not just because it’s very personal, but also because it has been the hardest few months in my life. The decisions I faced were far more consequential and way harder to grapple than any decisions I had to make as a leader or any time during my professional career. However I feel that other people — people who will never go through anything like I did — can learn from my story a lot exactly because it has been a very unusual and difficult problem to solve."

uk: вхід без авторизації

name: What transitioning from male to female taught me about leadership


When gRPC met Python

uk: Комп'ютерні науки

description: gRPC is one of the most new breakthroughs in the world of client server interaction. Using gRPC our client can directly make a call to a server on a different machine as if it were a local object. gRPC has low latency, high scalability and supports multiple use cases for distributed system. We can even build mobile clients which can communicate to a cloud server. gRPC uses Protocol Buffers which is an open source mechanism for serializing structured data, which makes payloads faster, smaller and simpler. In this talk we will try to understand how can we get started with gRPC in Python. grpcio package of python will be used for the demonstration of the examples and we will cover basics of gRPC as well. We will build a basic gRPC service and define protocol buffers for it. Demonstration of how a client and a server can be made through gRPC and how can they communicate.

uk: вхід без авторизації


When Models Query Models

uk: Комп'ютерні науки

description: The design of large-scale engineering systems, including but not limited to aerospace, particle accelerators, nuclear power plants, is carried out by a wide range of numerical models such as CAD files, finite-element models, and machine learning surrogate models to name a few. In order to provide a uniform modelling interface, we encapsulate numerical models in notebooks. A notebook is controlling model creation, execution, and query of results. Numerical solvers are embedded into Docker containers and provide an isolated and reproducible environment exposing a language-agnostic REST API. A model registry enables efficient queries of models. The overall system is represented as a collection of models that exchange data. Then, the design optimization involves execution of a dependency tree of models to study the impact of a parameter change and perform its optimization. In this contribution, we present a model query mechanism allowing notebook models to query one another. The model dependencies are represented with a graph with suitable processing algorithms. In order to ensure that only affected models are executed we derive and cache a model resolution order. The presented modelling framework relies on open source-technologies (packages: pydantic, Fast API, Jupyter, papermill, scrapbook, containers: Docker and Openshift as well as databases: MongoDB and Redis) and the talk will focus on good practices and design decisions encountered in the process."

uk: вхід без авторизації


uk: Комп'ютерні науки

description: Have you ever found yourself coding variations of a loop construct where fragments of the loop code were exactly the same between the variations? Or, in an attempt to factor out these common parts, you ended up with a loop construct containing a lot of conditional code for varying start, stop, or selection criteria? You might have felt that the end result just didn't look right. Because of the duplicated parts in your code, you noticed that the code didn't conform to the DRY (_Don't Repeat Yourself_) principle. Or, after an attempt to combine the variations into a single loop, with consequently a lot of conditional code, your inner voice told you that the resulting code had become too complex and difficult to maintain. This talk will show you a way out of this situation. It demonstrates how you can create a *generator function* that implements only the common parts of your loop construct. Subsequently you will learn how you can combine this generator function with distinct hand-crafted functions or building blocks from the standard library `itertools` module or the `more-itertools` package. As an example, imagine you'd need to implement some varying functionality based on the Fibonacci sequence. This talk shows you how it would look like before and after you've refactored it into a *pipeline of generators*. After having seen this pattern, you will recognize more quickly when this kind of refactoring helps you to create more maintainable and more Pythonic code.

uk: вхід без авторизації

name: When to refactor your code into generators and how


uk: Комп'ютерні науки

description: You have a performance problem, and you don't know what to do. All you know is that one of your endpoints or applications is too slow; and perhaps it only affects a certain user or customer. How do you figure out why it's slow, and what can you do to catch performance problems before they hurt users in production? We'll go through a wide range of strategies for detecting and diagnosing performance problems in typical production workloads. We'll cover both web-based domains as well as backend domains and other analytical applications involving number-crunching and big-data applications. We'll step through the following high-level strategies: - Tracing: through instrumentation of your code, you will get detailed traces of where the time is spent in generating your web server responses. - Profiling: we'll look at profiling strategies using both the Python built-in cProfile tool, as well as awesome 3rd party libraries like pyspy, including how to use these with pytest - Isolation: how to figure out if performance is affected by CPU, or memory, disk, or network IO limitations. - Reasoning: we'll look at common scenarios that result in performance regressions such as the needless execution of sub-queries in rendering web views, or algorithmic analysis and ""big-O"" notation, or concurrency problems resulting from exhaustion of threads in a pool and asyncio concurrency limitations resulting from overloaded subscription. - Prophylaxis: we'll look at how to include benchmarks within your CI pipeline, including with pytest and other technologies to catch performance regressions ahead of time.

uk: вхід без авторизації

name: Why is it slow? Strategies for solving performance problems


uk: Комп'ютерні науки

description: Attacks on software repositories are extremely common and can have a vast impact. A single successful compromise of the content distribution infrastructure can affect millions of users, voluntarily installing the infected packages. PEP 458 was designed to protect PyPI against a variety of possible attacks on PyPIs own content distribution network and PyPI mirrors, while giving administrators a mechanism to recover from a compromise if it happens. In addition, PEP 458 is a fundamental stepping stone for more advanced protection described in PEP 480. Both PEP 458 and 480 implement a specification called ""The Update Framework"" (TUF), which introduces a series of roles, keys and metadata formats that are published along with the packages they protect, and can be verified by a client software such as pip. Over the past couple of months we have made an effort to integrate the latest version of the Python TUF reference implementation with PyPI/Warehouse; see draft PR. In this talk we will give an introduction to PEP 458 and TUF, how it works and what it is good for. We will report from the work-in-progress integration with Warehouse, what challenges we face and how Python developer and user workflows are affected, as well as an expected timeline for the integration. And last but not least, we want to give an outlook of what comes after PEP 458, that is full developer-to-user end-to-end protection of Python packages as described by PEP 480. With our talk we also hope to spark interest in software supply chain security and to encourage the community to get involved by reviewing, commenting and contributing to the PEP 458 and PEP 480 integration efforts.

uk: вхід без авторизації

name: WIP: Implementing PEP 458 to Secure PyPI downloads