uk: Комп'ютерні науки

description: Subgraph matching finds all distinct isomorphic embeddings of a query graph on a data graph. For large graphs, current solutions face the scalability challenge due to expensive joins, excessive false candidates, and workload imbalance. In this paper, we propose a novel framework for subgraph listing based on Compact Embedding Cluster Index (idx), which divides the data graph into multiple embedding clusters for parallel processing. The sub has three unique techniques: utilizing the BFS-based filtering and reverse-BFS-based refinement to prune the unpromising candidates early on, replacing the edge verification with set intersection to speed up the candidate verification, and using search cardinality based cost estimation for detecting and dividing large embedding clusters in advance. The experiments performed on several real and synthetic datasets show that the sub outperforms state-of-the-art solutions on average by 20.4times for listing all embeddings and by 2.6times for enumerating the first 1,024 embeddings.

uk: вхід без авторизації

name: CECI: Compact Embedding Cluster Index for Scalable Subgraph Matching


Distance-generalized Core Decomposition

Francesco Bonchi

type: Person

cs: Informatika

de: Informatik

fi: tietojenkäsittelytiede

uk: Комп'ютерні науки

en: Computer Science

fr: Informatique

da: Datalogi

es: Ciencias de la Computación

nl: Informatica

cs: Technické vědy

de: Ingenieurwissenschaften

fi: tekniikka

uk: Інженерія

en: Engineering Sciences

fr: Ingénierie

da: Ingeniørvidenskab

es: Ingeniería

nl: Engineering

description: The k-core of a graph is defined as the maximal subgraph in which every vertex is connected to at least k other vertices within that subgraph. In this work we introduce a distance-based generalization of the notion of k-core, which we refer to as the (k,h)-core, i.e., the maximal subgraph in which every vertex has at least k other vertices at distance leq h within that subgraph. We study the properties of the (k,h)-core showing that it preserves many of the nice features of the classic core decomposition (e.g., its connection with the notion of distance-generalized chromatic number) and it preserves its usefulness to speed-up or approximate distance-generalized notions of dense structures, such as h-club. Computing the distance-generalized core decomposition over large networks is intrinsically complex. However, by exploiting clever upper and lower bounds we can partition the computation in a set of totally independent subcomputations, opening the door to top-down exploration and to multithreading, and thus achieving an efficient algorithm.

TIB AV-Portal

type: Service

type: WebContent

type: MediaObject

@language: en

cs: přihlášení není vyžadováno

de: keine Anmeldung erforderlich

uk: вхід без авторизації

en: no login required

nl: geen login vereist

ACM SIGMOD

type: Organization

cs: Video

de: Video

fi: Video

uk: Відео

en: Video

fr: Vidéo

da: Video

nl: Video

es: Video


uk: Комп'ютерні науки

description: Real-time data analysis and management are increasingly critical for today’s businesses. SQL is the de facto lingua franca for these endeavors, yet support for robust streaming analysis and management with SQL remains limited. Many approaches restrict semantics to a reduced subset of features and/or require a suite of non-standard constructs. Additionally, use of event timestamps to provide native support for analyzing events according to when they actually occurred is not pervasive, and often comes with important limitations. We present a three-part proposal for integrating robust streaming into SQL, namely: (1) time-varying relations as a foundation for classical tables as well as streaming data, (2) event time semantics, (3) a limited set of optional keyword extensions to control the materialization of time-varying query results. We show how with these minimal additions it is possible to utilize the complete suite of standard SQL semantics to perform robust stream processing. We motivate and illustrate these concepts using examples and describe lessons learned from implementations in Apache Calcite, Apache Flink, and Apache Beam. We conclude with syntax and semantics of a concrete proposal for extensions of the SQL standard and note further areas of exploration.

uk: вхід без авторизації

name: Apache Hive: From MapReduce to Enterprise-grade Big Data Warehousing


uk: Комп'ютерні науки

description: We introduce a wait-free, multi-reader, single-writer, kill -9 durable, indexing structure for in-memory social graph databases. This structure requires no communication from the readers back to the writer, allowing for trivial read scalability and isolation. We support online updates without compromising availability or read performance. Our structure supports looking up small subgraphs in 80 nanoseconds and a materialization rate of 12 nanoseconds per edge. Storage takes 7 bytes per edge per index and supports almost 1 million online writes per second.

uk: вхід без авторизації

name: Nanosecond Indexing of Graph Data With Hash Maps and VLists


uk: Комп'ютерні науки

description: An appropriate set of indexes can result in orders of magnitude better query performance. Index management is a challenging task even for expert human administrators. Fully automating this process is of significant value. We describe the challenges, architecture, design choices, implementation, and learnings from building an industrial-strength auto-indexing service for Microsoft Azure SQL Database, a relational database service. Our service has been generally available for more than two years, generating index recommendations for every database in Azure SQL Database, automatically implementing them for a large fraction, and significantly improving performance of hundreds of thousands of databases. We also share our experience from experimentation at scale with production databases which gives us confidence in our index recommendation quality for complex real applications.

uk: вхід без авторизації

name: Automatically Indexing Millions of Databases in Microsoft Azure SQL Database


uk: Комп'ютерні науки

description: In this paper we propose Fractal, a high performance and high productivity system for supporting distributed graph pattern mining (GPM) applications. Fractal employs a dynamic (auto-tuned) load-balancing based on a hierarchical and locality-aware work stealing mechanism, allowing the system to adapt to different workload characteristics. Additionally, Fractal enumerates subgraphs by combining a depth-first strategy with a from scratch processing paradigm to avoid storing large amounts of intermediate state and, thus, improves memory efficiency. Regarding programmer productivity, Fractal presents an intuitive, expressive and modular API, allowing for rapid compositional expression of many GPM algorithms. Fractal-based implementations outperform both existing systemic solutions and specialized distributed solutions on many problems - from frequent graph mining to subgraph querying, over a range of datasets.

uk: вхід без авторизації

name: Fractal: A General-Purpose Graph Pattern Mining System


uk: Комп'ютерні науки

description: Discovering interesting data patterns is a common and important analytical need in data, with increasing user demand for automated discovery abilities. However, automatically discovering interesting patterns from multi-dimensional data remains challenging. Existing techniques focus on mining individual types of patterns. There is a lack of unified formulation for different pattern types, as well as general mining frameworks to derive them effectively and efficiently. We present a novel technique QuickInsights, which quickly and automatically discovers interesting patterns from multi-dimensional data. QuickInsights proposes a unified formulation of interesting patterns, called insights, and designs a systematic mining framework to discover high-quality insights efficiently. We demonstrate the effectiveness and efficiency of QuickInsights through our evaluation on 447 real datasets as well as user studies on both expert users and non-expert users. QuickInsights is released in Microsoft Power BI.

uk: вхід без авторизації

name: QuickInsights: Quick and Automatic Discovery of Insights from Multi-Dimensional Data


uk: Комп'ютерні науки

description: Certain answers are a principled method for coping with uncertainty that arises in many practical data management tasks. Unfortunately, this method is expensive and may ex- clude useful (if uncertain) answers. Thus, users frequently resort to less principled approaches to resolve uncertainty. In this paper, we propose Uncertainty Annotated Databases (UA-DBs), which combine an under- and over-approximation of certain answers to achieve the reliability of certain answers, with the performance of a classical database system. Furthermore, in contrast to prior work on certain answers, UA-DBs achieve a higher utility by including some (explicitly marked) answers that are not certain. UA-DBs are based on incomplete K-relations, which we introduce to generalize the classical set-based notion of incomplete databases and certain answers to a much larger class of data models. Using an implementation of our approach, we demonstrate experimentally that it efficiently produces tight approximations of certain answers that are of high utility

uk: вхід без авторизації

name: Uncertainty Annotated Databases - A Lightweight Approach for Approximating Certain Answers


uk: Комп'ютерні науки

description: Entity matching (EM) finds data instances that refer to the same real-world entity. In 2015, we started the Magellan project at UW-Madison, joint with industrial partners, to build EM systems. Most current EM systems are stand-alone monoliths. In contrast, Magellan borrows ideas from the field of data science (DS), to build a new kind of EM systems, which is an ecosystem of interoperable tools. This paper provides a progress report on the past 3.5 years of Magellan, focusing on the system aspects and on how ideas from the field of data science have been adapted to the EM context. We argue why EM can be viewed as a special class of DS problems, and thus can benefit from system building ideas in DS. We discuss how these ideas have been adapted to build pymatcher and cloudmatcher, EM tools for power users and lay users. These tools have been successfully used in 21 EM tasks at 12 companies and domain science groups, and have been pushed into production for many customers. We report on the lessons learned, and outline a new envisioned Magellan ecosystem, which consists of not just on-premise Python tools, but also interoperable microservices deployed, executed, and scaled out on the cloud, using tools such as Dockers and Kubernetes.

uk: вхід без авторизації

name: Entity Matching Meets Data Science: A Progress Report from the Magellan Project


uk: Комп'ютерні науки

description: Speeding up probabilistic inference remains a key challenge in probabilistic databases (PDBs) and the related area of statistical relational learning (SRL). Since computing probabilities for query answers is #P-hard, even for fairly simple conjunctive queries, both the PDB and SRL communities have proposed a number of approximation techniques over the years. The two prevalent techniques are either (i) MCMC-style sampling or (ii) branch-and-bound (B&B) algorithms that iteratively improve model-based bounds using a combination of variable substitution and elimination. We propose a new anytime B&B approximation scheme that encompasses all prior model-based approximation schemes proposed in the PDB and SRL literature. Our approach relies on the novel idea of “scaled dissociation” which can improve both the upper and lower bounds of existing modelbased algorithms. We apply our approach to the well-studied problem of evaluating self-join-free conjunctive queries over tuple-independent PDBs, and show a consistent reduction in approximation error in our experiments on TPC-H, Yago3, and a synthetic benchmark setting.

uk: вхід без авторизації

name: Anytime Approximation in Probabilistic Databases via Scaled Dissociations


uk: Комп'ютерні науки

description: Query autocompletion (QAC) is an important interactive feature that assists users in formulating queries and saving keystrokes. Due to the convenience it brings to users, QAC has been adopted in many applications, including Web search engines, integrated development environments (IDEs), and mobile devices. For existing QAC methods, users have to manually type delimiters to separate keywords in their inputs. In this paper, we propose a novel QAC paradigm through which users may abbreviate keywords by prefixes and do not have to explicitly separate them. Such paradigm is useful for applications where it is inconvenient to specify delimiters, such as desktop search, text editors, and input method editors. E.g., in an IDE, users may input getnev and we suggest GetNextValue. We show that the query processing method for traditional QAC, which utilizes a trie index, is inefficient under the new problem setting. A novel indexing and query processing scheme is hence proposed to efficiently complete queries. To suggest meaningful results, we devise a ranking method based on a Gaussian mixture model, taking into consideration the way in which users abbreviate keywords, as opposed to the traditional ranking method that merely considers popularity. Efficient top-k query processing techniques are developed on top of the new index structure. Experiments demonstrate the effectiveness of the new QAC paradigm and the efficiency of the proposed query processing method.

uk: вхід без авторизації

name: Autocompletion for Prefix-Abbreviated Input


uk: Комп'ютерні науки

description: Alibaba runs the largest e-commerce platform in the world serving more than 600 million customers, with a GMV (gross merchandise value) exceeding USD 768 billion in FY2018. Online e-commerce transactions have three notable characteristics: (1) drastic increase of transactions per second with the kickoff of major sales and promotion events, (2) a large number of hot records that can easily overwhelm system buffers, and (3) quick shift of the 'temperature' (hot v.s. warm v.s. cold) of different records due to the availability of promotions on different categories over different short time periods. For example, Alibaba's OLTP database clusters experienced a 122 times increase of transactions on the start of the Singles' Day Global Shopping Festival in 2018, processing up to 491,000 sales transactions per second which translate to more than 70 million database transactions per second. To address these challenges, we introduce X-Engine, a write-optimized storage engine of POLARDB built at Alibaba, which utilizes a tiered storage architecture with the LSM-tree (log-structured merge tree) to leverage hardware acceleration such as FPGA-accelerated compactions, and a suite of optimizations including asynchronous writes in transactions, multi-staged pipelines and incremental cache replacement during compactions. Evaluation results show that X-Engine has outperformed other storage engines under such transactional workloads.

uk: вхід без авторизації

name: X-Engine: An Optimized Storage Engine for Large-scale E-commerce Transaction Processing


uk: Комп'ютерні науки

description: Visual graph query interfaces (a.k.a textscgui) widen the reach of graph querying frameworks across different users by enabling non-programmers to use them. Consequently, several commercial and academic frameworks for querying a large collection of small- or medium-sized data graphs (e.g., chemical compounds) provide such visual interfaces. Majority of these interfaces expose a fixed set of canned patterns (i.e., small subgraph patterns) to expedite query formulation by enabling pattern-at-a-time in lieu of edge-at-a-time construction mode. Canned patterns to be displayed on a textscgui are typically selected manually based on domain knowledge. However, manual generation of canned patterns is labour intensive. Furthermore, these patterns may not sufficiently cover the underlying data graphs to expedite visual formulation of a wide range of subgraph queries. In this paper, we present a generic and extensible framework called textscCatapult to address these limitations. textscCatapult takes a data-driven approach to automatically select canned patterns, thereby taking a concrete step towards the vision of data-driven construction of visual query interfaces. Specifically, it first clusters the underlying data graphs based on their topological similarities and then summarize each cluster to create a cluster summary graph (textsccsg). The canned patterns within a user-specified pattern budget are then generated from these textsccsgs by maximizing coverage and diversity, and minimizing cognitive load of the patterns. Experimental study with real-world datasets and visual graph interfaces demonstrates the superiority of textscCatapult compared to traditional techniques.

uk: вхід без авторизації

name: CATAPULT: Data-driven Selection of Canned Patterns for Efficient Visual Graph Query Formulation


uk: Комп'ютерні науки

uk: вхід без авторизації

name: ExplainIt! – A Declarative Root-cause Analysis Engine for Time Series Data


uk: Комп'ютерні науки

description: Modern search engines provide contextual information surrounding query entities beyond ten blue links in the form of information cards. Among the various attributes displayed about entities there has been recent interest in providing fun facts. Obtaining such trivia at a large scale is, however, non-trivial: hiring professional content creators is expensive and extracting statements from the Web is prone to uninteresting, out-of-context and/or unreliable facts. In this paper we show how fun facts can be mined from superlative tables in Wikipedia, whose rows are ranked according to some statistics, to provide a large volume of reliable and interesting content. We employ a template-based approach to semi-automatically generate natural language statements as fun facts. We show how to bootstrap and streamline the process for faster and cheaper task completion. However, the content contained in these tables is dynamic. Therefore, we address the problem of automatically maintaining the pairing of templates to tables as the tables are updated over time. Fun facts produced by our work is now part of Google's production search results.

uk: вхід без авторизації

name: Automatically Generating Interesting Facts from Wikipedia Tables


uk: Комп'ютерні науки

description: Distance labeling approaches are widely adopted to speed up the online performance of shortest distance queries. The construction of the distance labeling, however, can be exhaustive especially on big graphs. For a major category of large graphs, small-world networks, the state-of-the-art approach is Pruned Landmark Labeling (PLL). PLL prunes distance labels based on a node order and directly constructs the pruned labels by performing breadth-first searches in the node order. The pruning technique, as well as the index construction, has a strong sequential nature which hinders PLL from being parallelized. It becomes an urgent issue on massive small-world networks whose index can hardly be constructed by a single thread within a reasonable time. This paper scales distance labeling on small-world networks by proposing a Parallel Shortest-distance Labeling (PSL) scheme and further reducing the index size by exploiting graph and label properties. PSL insightfully converts the PLL's node-order dependency to a shortest-distance dependence, which leads to a propagation-based parallel labeling in D rounds where D denotes the diameter of the graph. Extensive experimental results verify our efficiency on billion-scale graphs and near-linear speedup in a multi-core environment

uk: вхід без авторизації

name: Scaling Distance Labeling on Small-World Networks


uk: Комп'ютерні науки

description: We propose CFS, a distributed file system for large scale container platforms. CFS supports both sequential and random file accesses with optimized storage for both large files and small files, and adopts different replication protocols for different write scenarios to improve the replication performance. It employs a metadata subsystem to store and distribute the file metadata across different storage nodes based on the memory usage. This metadata placement strategy avoids the need of data rebalancing during capacity expansion. CFS also provides POSIX-compliant APIs with relaxed semantics and metadata atomicity to improve the system performance. We performed a comprehensive comparison with Ceph, a widely-used distributed file system on container platforms. Our experimental results show that, in testing 7 commonly used metadata operations, CFS gives around 3 times performance boost on average. In addition, CFS exhibits better random-read/write performance in highly concurrent environments with multiple clients and processes.

uk: вхід без авторизації

name: CFS: A Distributed File System for Large Scale Container Platforms


uk: Комп'ютерні науки

description: We report a systematic performance study of streaming graph partitioning algorithms. Graph partitioning plays a crucial role in overall system performance as it has a significant impact on both load balancing and inter- machine communication. The streaming model for graph partitioning has recently gained attention due to its ability to scale to very large graphs with limited resources. The main objective of this study is to understand how the choice of graph partitioning algorithm affects system performance, resource usage and scalability. We focus on both offline graph analytics and online graph query workloads. The study considers both edge-cut and vertex-cut approaches. Our results show that the no partitioning algorithms performs best in all cases, and the choice of graph partitioning algorithm depends on: (i) type and degree distribution of the graph, (ii) characteristics of the workloads, and (iii) specific application requirements

uk: вхід без авторизації

name: Experimental Analysis of Streaming Algorithms for Graph Partitioning


uk: Комп'ютерні науки

description: A similarity join combines vectors based on a distance condition. Typically, such algorithms apply a filter step (by indexing or sorting) and then refine pairs of candidate vectors. In this paper, we propose to refine the pairs in an order defined by a space-filling curve which dramatically improves data locality. Modern multi-core microprocessors are supported by a deep memory hierarchy including RAM, various levels of cache, and registers. The space-filling curve makes our proposed algorithm cache-oblivious to fully exploit the memory hierarchy and to reach the possible peak performance of a multi-core processor. Our novel space-filling curve called Fast General Form (FGF) Hilbert solves a number of limitations of well-known approaches: it is non-recursive, it is not restricted to traverse squares, and it has a constant time and space complexity. As we demonstrate the easy transformation from conventional into cache-oblivious loops we believe that many algorithms for complex joins and other database operators could be transformed systematically into cache-oblivious SIMD and MIMD parallel algorithms.

uk: вхід без авторизації

name: Cache-oblivious High-performance Similarity Join


uk: Комп'ютерні науки

description: Graph processing on GPUs received much attention in the industry and the academia recently, as the hardware accelerator offers attractive potential for performance boost. However, the high-bandwidth device memory on GPUs has limited capacity that constrains the size of the graph to be loaded on chip. In this paper, we introduce GPU-based graph traversal on compressed graphs, so as to enable the processing of graphs having a larger size than the device memory. Designed towards GPU’s SIMT architecture, we propose two novel parallel scheduling strategies Two-Phase Traversal and Task-Stealing to handle thread divergence and workload imbalance issues when decoding the compressed graph. We further optimize our solution against power-law graphs by proposing Warp-centric Decoding and Residual Segmentation to facilitate parallelism on processing skewed out-degree distribution. Extensive experiments show that with 2x-18x compression rate, our proposed GPU-based graph traversal on compressed graphs (GCGT) achieves competitive efficiency compared with the state-of-the-art graph traversal approaches on non-compressed graphs.

uk: вхід без авторизації

name: GPU-based Graph Traversal on Compressed Graphs


uk: Комп'ютерні науки

description: As a dual problem of influence maximization, the seed minimization problem asks for the minimum number of seed nodes to influence a required number eta of users in a given social network G. Existing algorithms for seed minimization mostly consider the it non-adaptive setting, where all seed nodes are selected in one batch without observing how they may influence other users. In this paper, we study seed minimization in the it adaptive setting, where the seed nodes are selected in several batches, such that the choice of a batch may exploit information about the actual influence of the previous batches. We propose a novel algorithm, it ASTI, which addresses the adaptive seed minimization problem in OBig(fraceta cdot (m+n)varepsilon^2ln n Big) expected time and offers an approximation guarantee of frac(ln eta+1)^2(1 - (1-1/b)^b) (1-1/e)(1-varepsilon) in expectation, where eta is the targeted number of influenced nodes, b is size of each seed node batch, and varepsilon in (0, 1) is a user-specified parameter. To the best of our knowledge, ASTI is the first algorithm that provides such an approximation guarantee without incurring prohibitive computation overhead. With extensive experiments on a variety of datasets, we demonstrate the effectiveness and efficiency of ASTI over competing methods.

uk: вхід без авторизації

name: Efficient Approximation Algorithms for Adaptive Seed Minimization


Interactive Graph Search

uk: Комп'ютерні науки

description: We study interactive graph search (IGS), with the conceptual objective of departing from the conventional top-down strategy in searching a poly-hierarchy, a.k.a. a decision graph. In IGS, a machine assists a human in looking for a target node z in an acyclic directed graph G, by repetitively asking questions. In each question, the machine picks a node u in G, asks a human is there a path from u to z?', and takes a boolean answer from the human. The efficiency goal is to locate z with as few questions as possible. We describe algorithms that solve the problem by asking a provably small number of questions, and establish lower bounds indicating that the algorithms are optimal up to a small additive factor. An experimental evaluation is presented to demonstrate the usefulness of our solutions in real-world scenarios.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: MongoDB is a distributed database that supports replication and horizontal partitioning (sharding). MongoDB replica sets consist of a primary that accepts all client writes and then propagates those writes to the secondaries. Each member of the replica set contains the same set of data. For horizontal partitioning, each shard (or partition) is a replica set. This paper discusses the design and rationale behind MongoDB’s implementation of a cluster- wide logical clock and causal consistency. The design leveraged ideas from across the research community to ensure that the implementation adds minimal processing overhead, tolerates possible operator errors, and gives protection against non-trusted client attacks. While the goal of the team was not to discover or test new algorithms, the practical implementation necessitated a novel combination of ideas from the research community on causal consistency, security, and minimal performance overhead at scale. This paper describes a large scale, practical implementation of causal consistency using a hybrid logical clock, adding the signing of logical time ranges to the protocol, and introducing performance optimizations necessary for systems at scale. The implementation seeks to define an event as a state change and as such must make forward progress guarantees even during periods of no state changes for a partition of data.

uk: вхід без авторизації

name: Implementation of Cluster-wide Logical Clock and Causal Consistency in MongoDB


uk: Комп'ютерні науки

description: it SimRank is a classic measure of the similarities of nodes in a graph. Given a node u in graph G =(V, E), a single-source SimRank query returns the SimRank similarities s(u, v) between node u and each node v in V. This type of queries has numerous applications in web search and social networks analysis, such as link prediction, web mining, and spam detection. Existing methods for single-source SimRank queries, however, incur query cost at least linear to the number of nodes n, which renders them inapplicable for real-time and interactive analysis. This paper proposes prsim, an algorithm that exploits the structure of graphs to efficiently answer single-source SimRank queries. prsim uses an index of size O(m), where m is the number of edges in the graph, and guarantees a query time that depends on the reverse PageRank distribution of the input graph. In particular, we prove that prsim runs in sub-linear time if the degree distribution of the input graph follows the power-law distribution, a property possessed by many real-world graphs. Based on the theoretical analysis, we show that the empirical query time of all existing SimRank algorithms also depends on the reverse PageRank distribution of the graph. Finally, we present the first experimental study that evaluates the absolute errors of various SimRank algorithms on large graphs, and we show that prsim outperforms the state of the art in terms of query time, accuracy, index size, and scalability.

uk: вхід без авторизації

name: PRSim: Sublinear Time SimRank Computation on Large Power-Law Graphs


uk: Комп'ютерні науки

description: Given an undirected graph G and a seed node s, the local clustering problem aims to identify a high-quality cluster containing s in time roughly proportional to the size of the cluster, regardless of the size of G. This problem finds numerous applications on large-scale graphs. Recently, heat kernel PageRank (HKPR), which is a measure of the proximity of nodes in graphs, is applied to this problem and found to be more efficient compared with prior methods. However, existing solutions for computing HKPR either are prohibitively expensive or provide unsatisfactory error approximation on HKPR values, rendering them impractical especially on billion-edge graphs. In this paper, we present TEA and TEA+, two novel local graph clustering algorithms based on HKPR, to address the aforementioned limitations. Specifically, these algorithms provide non-trivial theoretical guarantees in relative error of HKPR values and the time complexity. The basic idea is to utilize deterministic graph traversal to produce a rough estimation of exact HKPR vector, and then exploit Monte-Carlo random walks to refine the results in an optimized and non-trivial way. In particular, TEA+ offers practical efficiency and effectiveness due to non-trivial optimizations. Extensive experiments on real-world datasets demonstrate that TEA+ outperforms the state-of-the-art algorithm by more than four times on most benchmark datasets in terms of computational time when achieving the same clustering quality, and in particular, is an order of magnitude faster on large graphs including the widely studied Twitter and Friendster datasets.

uk: вхід без авторизації

name: Efficient Estimation of Heat Kernel PageRank for Local Clustering


PS2: Parameter Server on Spark

uk: Комп'ютерні науки

description: Most of the data is extracted and processed by Spark in Tencent Machine Learning Platform. However, seldom of them use Spark MLlib, an official machine learning (ML) library on top of Spark due to its inefficiency. In contrast, systems like parameter servers, XGBoost and TensorFlow are more used, which incur expensive cost of transferring data in and out of Spark ecosystem. In this paper, we identify the causes of inefficiency in Spark MLlib and solve the problem by building parameter servers on top of Spark. We propose PS2, a parameter server architecture that integrates Spark without hacking the core of Spark. With PS2, we leverage the power of Spark for data processing and ML training, and parameter servers for maintaining ML models. By carefully analyzing Tencent ML workloads, we figure out a widely existing computation pattern for ML models-element-wise operations among multiple high dimensional vectors. Based on this observation, we propose a new data abstraction, called Dimension Co-located Vector (DCV) for efficient model management in PS2. A DCV is a distributed vector that considers locality in parameter servers and enables efficient computation with multiple co-located distributed vectors. For ease-of-use, we also design a wide variety of advanced operators for operating DCVs. Finally, we carefully implement the PS2 system and evaluate it against existing systems on both public and Tencent workloads. Empirical results demonstrate that PS2 can outperform Spark MLlib by up to 55.6X and specialized ML systems like Petuum by up to 3.7X.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: Due to the ubiquity of graphs, graph analytics has attracted much attention from both research and industry communities. The notion of k-truss is widely used in graph analytics. Since graphs are continuously evolving in real applications and it is costly to compute trusses from scratch, we study the problem of truss maintenance which aims at designing efficient incremental algorithms to update trusses when graphs are updated with changes. An incremental algorithm is desired to be bounded; that is, its cost is of O(f(|CHANGED|_c)) for some polynomial function f and some positive integer c, where CHANGED comprises the changes to both the graph and the result and |CHANGED|_c is the size of the c-hop neighborhood of CHANGED. An incremental problem is bounded if it has a bounded incremental algorithm and is unbounded otherwise. Under the model of locally persistent algorithms, we prove that truss maintenance is bounded under edge removals but is unbounded even for unit edge insertions. To address the unboundedness, we formulate a new notion AFF^preceq which, as a practically effective alternative to CHANGED, represents a set of edges affected by the changes to the graph, and devise an insertion algorithm that is bounded with respect to AFF^preceq, while retaining the boundedness for edge removals. More specifically, our insertion algorithm runs in O(f(|AFF^preceq|_c)) time for some polynomial function f and some positive integer c with |AFF^preceq|_c being the size of the c-hop neighborhood of AFF^preceq. Our extensive performance studies show that our new algorithms can significantly outperform the state-of-the-art by up to 3 orders of magnitude for the 12 large real graphs tested and are more efficient than computing trusses from scratch even for changes of non-trivial size. We report our findings in this paper.

uk: вхід без авторизації

name: Unboundedness and Efficiency of Truss Maintenance in Evolving Graphs


Grundlagen des Fahrlässigkeitsunrechts

uk: Комп'ютерні науки

de: Rechtswissenschaften

uk: Правознавство

en: Law

de: Rechts-, Wirtschafts- und Sozialwissenschaften

uk: Право, економіка та суспільні науки

en: Law, Economics and Social Sciences

@language: de

uk: вхід без авторизації

Leibniz Universität Hannover (LUH)

addressCountry: DE

addressLocality: Hanover

addressRegion: Lower Saxony


uk: Комп'ютерні науки

uk: вхід без авторизації

name: Cilium - Bringing the BPF Revolution to Kubernetes Networking and Security


uk: Комп'ютерні науки

description: Traceleft is a framework built upon eBPF which allows generation of system events such as file operations and network calls via a configuration driven system. It can act as a foundation for building auditing and incident analysis or monitoring tools that work at the system level and need targeted and filtered information. While traditionally, eBPF based analysis requires writing scripts in a limited C subset, Traceleft aims at providing a configuration based definition system events to be monitored over which filters can be set as required. This talk shares our experiences of developing Traceleft along with some use cases and challenges faced. The tiny eBPF VM in the Linux kernel has given us unprecedented flexibility in developing tooling for analyzing systems. In this talk we discuss Traceleft - a new auditing and incident analysis framework that can be used to build monitoring tools for filesystem and network events. Traceleft is a configuration driven framework that latches on to syscalls and other relevant Linux kernel functions identified in Traceleft. The eBPF programs are generated, compiled on the fly and inserted in the kernel, based on a set of JSON based configurations. Traceleft supports tracing network calls, file operations such as opening/closing of a target files, for a given set of processes and the aggregation of these events. Traceleft is built using Go and uses Gobpf library and allows the configurations to be sent over the wire using Protobuf format. Traceleft started as a joint effort between ShiftLeft and Kinvolk to develop an easy, repeatable way to provide a configurable way to audit syscalls and classify security events filtered on processes and files. The tools built over Traceleft can be used for live monitoring and alert generation. Implementing Traceleft presented many challenges. eBPF tracing programs need to be recompiled for each kernel version and updated for each application profile. It needs to keep track of new processes and match them with new applications in a race-free way. Reading information from different sources bring many race possibilities. Catching syscalls and their parameters bring limitations: matching specific syscalls with specific syscalls or remote hosts for network calls. We will discuss how we overcame some of the challenges.

uk: вхід без авторизації

name: Configuration Driven Event Tracing with Traceleft and eBPF: Use cases and implementation challenges


uk: Комп'ютерні науки

description: Since docker came on the scene in 2013, for many it was the first they'd heard of containers. They existed before, and an have exploded since then. Today if you intend on running containers in production it means at the kubernetes level, though what is happening below that? what does this mean for direct access and on-going developer experience?

uk: вхід без авторизації

name: Container Runtimes: draw some lines: Future of connecting to the container runtime as docker phases out


uk: Комп'ютерні науки

description: A dive into the world of running systemd as an *in container* process manager at Facebook. At @FB we heavily utilize systemd on our servers. But, there's more! We also heavily utilize systemd inside our containers as well! Combined with our btrfs based image deployment mechanism we leverage all the good parts of systemd within our containers. I'll show how we utilize all the various components of systemd within a container, including how we use Portable Services! I'll talk about the philosophy of this design, our approach to building container images with systemd in mind, and our our approach to Runtime Composition of services. Come listen and enjoy a deep dive into how we use btrfs, systemd, and portable services, and what benefit it provides for us across our large container infrastructure.

uk: вхід без авторизації

name: Container Run-times and Fun-times: Using systemd inside a container @FB


CRI-O: All the Runtime Kubernetes need

uk: Комп'ютерні науки

description: CRI-O is a brand new container runtime dedicated and optimized to support kubernetes workload. Its goal is to be a stable container runtime tied to kubernetes releases, replacing the docker daemon. Historically every update of Docker has broken Kubernetes. This has led to major rewriting and fixes of Kubernetes, which is understandable since Docker is not primarily for Kubernetes. Kubernetes needs a container runtime dedicated to its specifications. CRI-O, the name comes from the Container Runtime Interface for Open container runtimes, takes advantages of emerging standards like OCI Runtime and Image Specification, as well as open source projects to handle container images (github.com:containers/image, github.com:containers/storage) . This means as these projects advance CRI-O will be able to take advantage of the improvements and features, but all the while guaranteeing that it will not break any functionality required by the Kubernetes CRI. CRI-O works with runc and Clear Containers runtimes. CRI-O was designed from the ground up to satisfy Kubernetes Container Runtime Interface, and currently passes all node and E2E tests. The github repository has been setup to not accept any pull requests that causes these tests to break. We will be tying the versions of CRI-O to the Kubernetes versions, to maintain complete compatibility. This talk will describe the CRI-O architecture as well as demonstrate different kubernetes features running on top of CRI-O exercising the CRI API. The attendees will learn how to configure CRI-O with kubernetes and use it for their workloads.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: dbus-broker is an implementation of the DBus specification, intended to be a drop-in replacement for the reference implementation on Linux. It is now scheduled to be the default system and user bus in the next Fedora release. This talks shows some of the lessons learned during this relatively young project, the challenges faced, what the current status is and the plans for the future.

uk: вхід без авторизації

name: dbus-broker: A Linux D-Bus Message Broker


Early boot provisioning systems

uk: Комп'ютерні науки

description: Early boot provisioning systems aim to enable automated, declarative, immutable patterns for Linux systems. In this talk, I'll discuss the CoreOS Ignition system and illustrate how it works and addresses real-world use cases on bare-metal, cloud providers, and hypervisors. I'll share experiences using and integrating early provisioning and highlight new challenges for contributors to explore. With Ignition shipping in several Linux distributions soon, early boot provisioning shows promise and potential for the future. Early boot provisioning systems prepare a Linux host by partitioning disks, creating filesystems, and writing systemd units and configs. They aim to enable automated, declarative, immutable infrastructure patterns and shape the way large-scale Linux systems operate. In this talk, I'll discuss the original CoreOS declarative config and Ignition system, share how it works, and how it arose. I'll walk through real-world use cases on bare-metal, cloud providers, and hypervisors to show early boot provisioning concepts and where they shine. I'll review experiences using Ignition and cloud-init and integrating them into Matchbox and the Typhoon Kubernetes distribution. For those designing or integrating early boot provisioners, I'll discuss advanced features, exciting areas for exploration, and my hopes for these systems over time. Now that Container Linux and Flatcar Linux support Ignition and Red Hat has announced Fedora CoreOS and Red Hat CoreOS will as well, early boot provisioning can evolve to benefit more distributions and users.

uk: вхід без авторизації


keywords: Sylvain Afchain

uk: Комп'ютерні науки

description: Efficient monitoring of large-scale networks poses a delicate balance between capture granularity on the one hand and the imposed overheads and performance penalties on the other. Skydive is an open source real-time network topology and protocol analyzer, featuring smart network collection which is both granular and efficient. Skydive allows for efficient network monitoring at scale through Linux networking features such as BPF and eBPF. In the talk we will present Skydive and will give an update of the features that introduced since one year. We will show how Skydive leverages eBPF to produce useful insights on top of network topology information. We will share some performance results showing the efficiency of Skydive BPF/eBPF capturing in the context of a Kubernetes deployment.

uk: вхід без авторизації

name: Efficient Network Analytics with BPF/eBPF using Skydive


Fearless Multimedia Programming

uk: Комп'ютерні науки

description: Whether you are interested in multimedia programming in specific or curious about how Rust programming language can enable you to write almost-bug-free code without having to compromise on efficiency, this talk is for you. GStreamer is a popular framework of choice for multimedia programming in the Linux world, especially for embedded. Since efficiency is a typical core requirement for embedded solutions, traditionally C/C++ have been the languages of choice for writing GStreamer applications and plugins. Unfortunately, this efficiency comes at the price of safety. Even the most skilled C/C++ developers make mistakes with memory management and the results could potentially be catastrophic. Thread safety is another aspect that is central to multimedia solutions but is extremely difficult to achieve with C/C++. Rust language is designed to be both efficient and safe at the same time. In this talk, Zeeshan will present how GStreamer's Rust bindings not only make multimedia programming a lot safer, easier and fun but also allow developers to write even more efficient code than C/C++ in certain cases. While the talk will be focused on multimedia and GStreamer in specific, most of the concepts presented shall be very generic.

uk: вхід без авторизації


Fedora CoreOS

uk: Комп'ютерні науки

description: What exactly is Red Hat up with CoreOS .....and what were they thinking when they announced a Fedora CoreOS? In this talk, we'll briefly look at some of the excellent work pioneered by the Container Linux team around the self-driving, container focused operating system. We'll also overlay how the container ecosystem has changed over the past 5 years and what we're doing at the OS-level to refocus and ultimately give users a better experience.

uk: вхід без авторизації


Flatpak, a technical walkthrough

uk: Комп'ютерні науки

description: Flatpak is a desktop-focused application distribution and deployment system for linux. This talk will walk through the technical details of the core functionallity and explain how it work and why it works that way. Flatpak has now had a major stable release and the rate of change is slowing down. This is a good time to take a look at how the things work behind the scenes. This talk will go through the technical details behind flatpak, explaining things like ostree, bubblewrap and how flatpak uses this to implement the sandbox and basic flatpak commands. It will also explain in high-level terms how the portals and other desktop integration features work.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: Logging could be considered simple when dealing with a few applications, but in environments with distributed systems where log data comes from multiple sources and likely in different formats it becomes complex and data analysis harder. In this session I will dig into the challenges of logging for distributed systems and the experience of building a tool called Fluent Bit to solve the problems associated with performance, unstructured data and log centralization within others.

uk: вхід без авторизації

name: Fluent Bit: Solving Logging Challenges for Cloud Native Environments


uk: Комп'ютерні науки

description: Uyuni, an opinionated fork of the Spacewalk project, provides open source lifecycle management for today's datacenter. With the help of Salt for configuration management it keeps your workloads up to date and secure. Uyuni manages all your Linux workloads. It bootstraps physical servers, creates VMs for virtualization and cloud, builds container images, and tracks what runs on your Kubernetes clusters. Uyuni is open source, backed by SUSE Linux, and actively developed. This presentation will give an overview about this project, it's current possibilities for managing datacenters, and briefly talk about future plans.

uk: вхід без авторизації

name: From Physical to Cloud to Container: One system to deploy and manage them all


uk: Комп'ютерні науки

description: This talk describes our experience developing Wormhole Connector, a distributed proxy component that connects external enterprise systems to a Kyma Kubernetes cluster. The connection between the Wormhole Connector and the Kubernetes cluster is based on HTTP/2, taking advantage of the stream concept to multiplex multiple connections (HTTP/1 or HTTP/2) through one active HTTP/2 connection. It will cover how we use the Serf and Raft libraries to make the Wormhole Connector highly available, our experience using the Go standard library to support HTTP/2 connections, and technical details describing how everything works under the hood. This talk describes our experience developing Wormhole Connector, a distributed proxy component that connects external enterprise systems to a Kyma Kubernetes cluster. The connection between the Wormhole Connector and the Kubernetes cluster is based on HTTP/2, taking advantage of the stream concept to multiplex multiple connections (HTTP/1 or HTTP/2) through one active HTTP/2 connection. It will cover how we use the Serf and Raft libraries to make the Wormhole Connector highly available, our experience using the Go standard library to support HTTP/2 connections, and technical details describing how everything works under the hood.

uk: вхід без авторизації

name: HTTP tunneling in Go using HTTP/2 streams: And making it highly available with Serf and Raft


Is Cockpit Secure?

uk: Комп'ютерні науки

description: Cockpit makes Linux discoverable. But it's really a Linux session in a web browser, accessing the native system APIs and tools directly from javascript. Does that sound scary? How can we be sure that accessing Linux from a web browser is secure? What about the web server stack? What about authentication and privilege escalation? We'll talk about how Cockpit deals with security, authentication, privilege escalation, and browser lock down. I'll show you various techniques to tailor Cockpit's security options to your situation, like using bastion hosts.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: Computer systems are complex. Most software applications are distributed and expected to scale. That does not make them any simpler. Further, there is the real world all of that is expected to work in. To analyze performance of such a system isn't less complex. Luckily, there is help. The Open Source universe is full of excellent tools that can help. The talk introduces a few of them. The talk walks down the path of comparing two Linux servers that look pretty much the same from the specs. Introducing test-tools like stress-ng, sysjitter, fio, packetdrill, and friends it is shown how to get a good overview of the systems CPU, memory, disk, and network performance. Since benchmark results are usually time variant a few basic statistical terms are refreshed and used for visualizing those results employing different chart types. The field of performance analysis is wide. There isn't more time in 45 minutes than just to scratch the surface. At the end of the talk the audience will have a few pointers of how to start evaluating their own Linux systems to provide the right performance for their applications.

uk: вхід без авторизації

name: Is my system fast?: An introduction of some tools that help to answer this question.


uk: Комп'ютерні науки

description: Kdump is a vital tool for debugging severe kernel crashes, especially if the failure can't be reproduced easily or an direct access to the system is not possible. When an sever error happens in the kernel, a new crash kernel get loaded which saves the memory of the crashed system. These dump can be used to analyze the state of the machine and hopefully give insights on what has happened. This talks will dive into the internals of kexec and kdump. How the crash kernel get set-up, how it's execution get triggered. We will also look into kexec-tool, the user-space part needed to set up a system to use kdump. Where necessary, the architectural specific details will be explained by looking at the arm64 implementation. This talk is thought for people who want to have an insight into how kdump is working. Kdump is a vital tool for debugging severe kernel crashes, especially if the failure can't be reproduced easily or an direct access to the system is not possible. When an sever error happens in the kernel, a new crash kernel get loaded which saves the memory of the crashed system. These dump can be used to analyze the state of the machine and hopefully give insights on what has happened. This talks will dive into the internals of kexec and kdump. How the crash kernel get set-up, how it's execution get triggered. We will also look into kexec-tool, the user-space part needed to set up a system to use kdump. Where necessary, the architectural specific details will be explained by looking at the arm64 implementation. This talk is thought for people who want to have an insight into how kdump is working.

uk: вхід без авторизації

name: Kexec/Kdump under the hood: How to debug your crashed system


uk: Комп'ютерні науки

description: libcapsule is a project that allows segregated dynamic linking: Access to the symbols of a library without being exposed to any of the dependencies of that library without requiring recompilation of the binary that pulls it in. libcapsule's goal is to improve portability of programs that are distributed with runtimes (cf flatpak) that still need access to [some] libraries from the host (eg libGL) while insulating said program from any system libraries outside the runtime other than those it directly requires.

uk: вхід без авторизації

name: libcapsule: segregated loading of dynamic libraries


uk: Комп'ютерні науки

description: Marco d'Iltri - Is it contained? Untrusted root users in containers Buildah

uk: вхід без авторизації

name: All Systems Go! 2018: Lightning Talks: Submit Your Talks at the Venue


uk: Комп'ютерні науки

description: In contrast to most firmware, like UEFI based or the BIOS, Linux is free software, has drivers for everything and is well known to the administrator. So why not use it in the firmware too? Some work has to be done though to fit it into the flash ROM chip and make it fast. Due to increases in flash ROM chip sizes, it’s possible again to put the Linux kernel and an initrd there, and use Linux as a boot loader. In 2017 the LinuxBoot project joined the Linux Foundation, and several big companies backed it. This talk presents the project, the benefits and gives a short demonstration in conjunction with coreboot. To achieve the goal, the Linux kernel should be adapted for the board to make it as small as possible. So, it’s back to building your own Linux kernel. Additionally, the boot time should not suffer, so methods on how to profile and to improve start-up time are presented.

uk: вхід без авторизації

name: LinuxBoot and booting fast: Use Linux as the bootloader and start the system up fast.


uk: Комп'ютерні науки

description: As we isolate functionality into services distributed across networks, we increasingly strain the concept of trust boundaries. Hosts are no longer simply trusted or untrusted, and each host comes with a new foothold for attackers. This risk is called the Confused Deputy Problem, and it’s part of a growing number of attacks, including the one on Equifax. We need to stop assuming trusted services -- especially huge ones like those in traditional web stacks -- remain trustworthy. Capability-based models offer hope, but we need a few more patterns to use them with modern microservices. The Confused Deputy Problem has a history going back to early Unix. Attacks on passwd are a classic case. The architecture of passwd is simple: allow unprivileged users to invoke a setuid executable that performs a high-privilege operation under defined preconditions. This means passwd has been deputized to alter the passwords for all users. However, if you can confuse passwd into misusing its ambient authority, you can gain privileges. As a rule, a compromised service can do anything the service itself is allowed to do. One of the answers has been Mandatory Access Control (MAC) like selinux. Firewalls are analogous for networked services. These systems operate by confining which principals can access which files or services. This helps somewhat; passwd has no business updating /boot (for example). However, in cases like Equifax, principals were only talking to the expected systems; the attacker used the web server to exfiltrate data from the database backing the web application, even accessing tables the web application would be expected to access. Just as with passwd, Equifax’s problem was trusting an application to both perform its functionality and enforce authorization. Capability-based models are one answer: rather than trusting a complex service to enforce authorization against deeper systems, we provide the client with proof that it’s allowed to access its own records and have the service forward this proof to deeper systems in order to read or manipulate corresponding records. It’s now harder to turn a compromised service into a system-wide attack. Systems using capabilities are in widespread use: Kerberos, Google’s Firebase, and concert tickets all use the concept of a token providing proof-of-authorization in a way decoupled from principals. However, things get weird when interaction is no longer directly between the primary principal (say, a web client) and nested services. Because possessing a capability is sufficient to exercise it, how do we handle cases where deeper services should have access to fewer (or more, or different) objects than the ones in front of them? In this presentation, we’ll look at state-of-the-art methods for delegating capabilities, including “sealing” (reducing the authorization of shallow services) and original techniques to forward only a subset of the caller’s capabilities (reducing the authorization of deep services), all designed for use with distributed services across networks. Note: The capabilities here are unrelated to Linux kernel capabilities. However, Linux does have some capabilities like the ones mentioned, including the idea of handing off a file descriptor from one application to another. This allows one app to open the FD and hand it to another that cannot (but can use the FD once in possession). Socket activating a web server on port 80 despite starting the web server as an unprivileged user is one example of handing off a capability via FD.

uk: вхід без авторизації

name: Little Services, Big Risks: Extending capability-based security models to achieve micro-segmentation for grids of services


uk: Комп'ютерні науки

description: In a distributed world, monitoring system calls with kauditd can present challenges. In this talk we will address some of those challenges and give a use case of how we build an event pipeline for monitoring file system events. With the rise of containers and generic container based operating systems we find ourselves with a large quantity of nodes that do general compute tasks. These nodes produce a large volume of audit data that we can leverage for many tasks. In our use case we wanted a way to monitor all file system changes in ways that we could not do with the existing libraries or tools. In this talk we will describe how we chose to use audit log system to monitor file system changes, how we built our system to scale and the pros and cons we have found from our solution. We will also talk about possible future work with respect to security and execution monitoring.

uk: вхід без авторизації

name: Monitoring File System Syscalls in a Distributed Architecture


uk: Комп'ютерні науки

description: Modern container engines such as systemd.nspawn and rkt provide a means of restricting privilege by limiting Linux capabilities. At Facebook, however, the heterogeneity of services and complexity of libraries running inside the container, along with our full init system model, make determining the set of capabilities that a task uses non-trivial. In this talk, we will discuss how we tackled this problem in a performant manner by building Capmond, a host-level daemon that leverages BPF to monitor capability usage by a process, and map it reliably to the associated container. Learn about the challenges we faced in making this work on our unique infrastructure, how this compares to known solutions such as auditd, and how we are leveraging Capmond to build sandboxes around capability usage, ssh sessions, system calls, and more. Capmond is in the process of being open sourced, so we'll also talk about how you can use it in your organization to help monitor your production systemd (nspawn) containers.

uk: вхід без авторизації

name: Monitoring Linux Capabilities in the Container using BPF


uk: Комп'ютерні науки

description: Sick of insecure PXE booting over TFTP? Come learn about our efforts to write modern boot loaders in Linux's user space. LinuxBoot is a project to put Linux kernels into firmware. LinuxBoot means we have the full toolset of modern languages like Go at our fingertips: let's use them! We can now easily develop boot loaders with trivial support for HTTPS-, gRPC- or TPM-based network booting.

uk: вхід без авторизації

name: Netboot21: Bootloaders in the 21st Century: User-space bootloaders with LinuxBoot


uk: Комп'ютерні науки

description: nettools is a yet-to-be-released project providing low-level libraries for network configuration. Within the scope of the project falls protocols like DHCP, NDP, IPv4 Address Conflict Detection, and IPv4LL. The first library scheduled to be released is IPv4ACD, a pre-release version of which is already used by NetworkManager. This talk gives an overview of the design principles of the project, the current status and the future plans.

uk: вхід без авторизації

name: nettools: A collection of network configuration libraries


oomd: A userspace OOM killer

uk: Комп'ютерні науки

description: Running out of memory on a host is a particularly nasty scenario. In the Linux kernel, if memory is being overcommitted, it results in the kernel out-of-memory (OOM) killer kicking in. In this talk, Daniel Xu will cover why the Linux kernel OOM killer is surprisingly ineffective and how oomd, a newly opensourced userspace OOM killer, does a more effective and reliable job. Not only does the switch from kernel space to userspace result a more flexible solution, but it also directly translates to better resource utilization. His talk will also do a deep dive into the Linux kernel changes and improvements necessary for oomd to operate. Out of memory killing has historically happened inside kernel space. On a memory overcommitted linux system, malloc(2) and friends will never fail. However, if an application dereferences the returned pointer and the system has run out of physical memory, the linux kernel is forced to OOM kill one or more processes. This is typically a slow and painful process because the kernel spends an unbounded amount of time swapping in and out pages and evicting the page cache. Furthermore, configuring policy is not very flexible while being somewhat complicated. oomd aims to solve this problem in userspace. oomd leverages cgroupsv2 and newly exposed counters and statistics to monitor a system holistically. oomd takes corrective action in userspace before an OOM occurs in kernel space. Corrective action is configured via a flexible plugin system, in which custom code can be written. By default, this involves killing offending processes. This enables an unparalleled level of flexibility where each workload can have custom protection rules. Furthermore, time spent churning pages in kernelspace is minimized. In practice at Facebook, we've regularly seen 30 minute host lockups go away entirely.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: A more generic approach to ensure you have what you'd expect to have A side effect of the many new ways to package filesystems (here's looking at you, containers!), is that filesystems are being copied around without many of the features that traditional packaging provided (i.e. `rpm -qV ...`). Much progress has been made for reproducible digests of containers. In this talk Vincent Batts will review options for distributing filesystems with reproducibility, and verifying the at-rest outcomes.

uk: вхід без авторизації

name: Passive filesystem verification: (and massage)


uk: Комп'ютерні науки

description: System containers, the oldest type of containers, focus on running an entire Linux distribution, including all its services in very much the same way it would on a physical system or virtual machine. System containers come with some unique challenges, users of those containers expect to be able to do pretty much everything that they can on a normal system. This means it’s not possible to restrict those containers quite as much as application containers can be. It also means that there are extra expectations to be met: Being able to add/remove devices to/from a running container Loading security profiles inside a container Using file capabilities in the container Mounting file systems Proper reporting of uptime, resource consumption and limits Live-migration In this presentation, we’ll explore some of the existing technologies in use by LXC and LXD to address some of those expectations as well as upcoming kernel and userspace features that will allow system containers to do even more than they do today.

uk: вхід без авторизації

name: Past, present and future of system containers


uk: Комп'ютерні науки

description: Portability is a shining goal for all software -- an objective since the beginning of computing through the present. And yet, it also remains illusive. We once sought to develop software "portable" between whole OSes and kernels; today, we've lost control of our build and distribution pipelines so completely that portability of a binary between two barely-different distributions of linux is considered radical and nearly impossible. Even "portability" of a binary when moving between two directories on the same host is often a trial by fire. In this talk, we will define and discuss two specific kinds of portability -- what it means to build path-agnostic libraries and what it means to support co-installable libraries -- then explore what this kind of freedom could do for us, and most importantly, practical ways to achieve these goals within existing systems. Computers are general purpose computers -- they can run any program. But theory aside, is that really true? Forget tivoization and the rise of walled-garden app stores -- when considering new software, often one of the first questions we ask ourselves as linux users is "Is it available on ${distro|packman}?"... We've already been doing this to ourselves for decades. Source builds are possible, but remain full of "well actually" moments; containerization hints at the sheer popularity of The Dream of escaping this madness; and yet, most of our day to day business is guided and limited by what's available in a package manager, and our machines as a whole are limited by which package manager we've chosen. Distributions become inherently balkanized: no one can install anyone else's packages with anything short of a full chroot. Why? Nobody *wants* this. The fracturing creeps in from somewhere. Is it something we can fix? What would it look like if we tried to define packages without distros? Packages without the need for management, beyond drag and drop? Packages with *bring your own* management? Easily relocatable, path-agnostic binaries, and co-installable libraries -- all of them able to work together, or be shipped independently at the user's choice, free of version conflicts in either form? We can have all these things. And if you think this is going to be a talk about static linking, think again: there are more options, and there are things we can do now, and we can do them within the ecosystems and infrastructure of existing distros non-disruptively. There are already a variety of approaches to producing relocatable binaries: from static linking, to snapshots and images which require chroots to be executable, to content-addressable (but still absolute-pathed) dynamic linking -- distros and tools using all of these approaches exist. In this talk, we'll survey all of them, plus go beyond into approaches to dynamic linking which are free from absolute paths completely. Finally, we'll compare and contrast all these approaches, map any limitations and assumptions, and try to plot a road ahead.

uk: вхід без авторизації

name: Path-agnostic binaries, co-installable libraries, and How To Have Nice Things


Peer to peer OS and flatpak updates

uk: Комп'ютерні науки

description: Recently, work that we have been doing on Endless OS to allow peer to peer OS and flatpak updates has been reaching maturity and nearing wider deployment. This talk will give an overview of how we support LAN and USB updates of OSTrees, how it fits in upstream in OSTree and flatpak, and what you’d need to do to enable peer to peer updates for your OSTree system.

uk: вхід без авторизації


Playing with casync @ instagram

uk: Комп'ютерні науки

description: In Instagram, we have been experimenting with casync as an alternative package format for deployment of the site. This talks describe our findings Last year when Lennart presented in ASG about casync, we were excited to check it out. So we checked what was necessary to deploy parts of our sites with it. and we spend some times experimenting... This talk show our results, so far...

uk: вхід без авторизації


Portable Services are Ready to Use

uk: Комп'ютерні науки

description: Portable Services bring certain aspects of containers to classic system service management. Let's discuss them in technical detail! Portable Services bring certain aspects of containers to classic system service management. With systemd v239 Portable Services are for the first time complete and ready for users to take advantage of. In this talk we'll have a look on the underlying technical concepts, how things fit together and what the precise limitations and benefits are.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: This talk will describe all of the reasons for podman, all of its features demonstrate its functionality, I will cover the background of podman, how we built it, why we built it, I will demonstrate using it in multiple different ways, Running containers building container images Communicating with it via var link, cockpit integration. Communicating with it from a remote machine.

uk: вхід без авторизації

name: Replacing Docker with Podman: Simple tool for running pods and containers


uk: Комп'ютерні науки

description: After years of development and experimentation, we finally have comprehensive OS-level work-conserving resource isolation working and are now in the process of deploying for various applications including workload protection and container stacking. This talk examines the project and the resulting resource control methods. FB has been actively experimenting with cgroup2 resource control for years. In the process, we developed several kernel and userland mechanisms, fixed numerous isolation issues, and discovered a number of surprising interactions. We finally have comprehensive OS-level work-conserving resource isolation working and are now in the process of refining and deploying the developed comprehensive resource isolation mechanism for various applications such as workload protection and container stacking. Let's take a look at the mistakes, the lessons, the result, and discuss how best this can be integrated into the whole operating system.

uk: вхід без авторизації

name: Resource Control @FB: Functional resource control using cgroup2


uk: Комп'ютерні науки

description: It is now possible to run Android ontop of an entirely Open Source Linux Graphics stack, this talk will dig into how you can do it too! Finally, it is possible to run Android on top of mainline Graphics! The recent addition of DRM Atomic Modesetting and Explicit Synchronization to the kernel paved the way, albeit some changes to the Android userspace were necessary. The Android graphics stack is built on a abstraction layer, thus drm_hwcomposer - a component to connect this abstraction layer to the mainline DRM API - was created. Moreover, changes to MESA and the abstraction layer itself were also needed for a full conversion to mainline. This talk will cover recent developments in the area which enabled Qualcomm, i.MX and Intel based platforms to run Android using the mainline graphics stack.

uk: вхід без авторизації

name: Running Android on the Mainline Graphics Stack


Scale Your Auditing Events

uk: Комп'ютерні науки

description: The Linux Audit daemon is responsible for writing audit records to the disk, which you can then access with ausearch and aureport. However, it turned out that parsing and centralizing these records is not as easy as you would hope. Elastic's new Auditbeat fixes this by keeping the original configuration, but ships them to a centralized location where you can easily visualize all events. You can also use Auditbeat to detect changes to critical files, like binaries and configuration files, and identify potential security policy violations. This talk shows you what can you do to discover changes, events, and potential security breaches as soon as possible on interactive dashboards. Additionally, we are combining Auditd events with logs, which are security relevant.

uk: вхід без авторизації


State of systemd @ Facebook

uk: Комп'ютерні науки

description: We'll be covering happenings, learnings and new challenges running and supporting systemd in production on the Facebook fleet throughout the past year. This talk is a followup to systemd @ Facebook — a year later that was presented last year. We'll cover the latest developments, how we're leveraging new systemd features, how we continue rolling systemd on the fleet, and finally discuss a number of interesting case studies.

uk: вхід без авторизації


systemd in 2018

uk: Комп'ютерні науки

uk: вхід без авторизації


The Future of Networking APIs

uk: Комп'ютерні науки

description: This talk presents TAPS (Transport Services), a proposed abstraction for a new Networking API, and calls for the Linux community to get involved. Even decades after its introduction, the Socket API is still the de-facto standard Networking API on many systems. Its simplifying abstraction of a network connection as a file helped making networking easier to integrate into applications in the early days, but by now, networking has become way more complex. We have to deal with asynchronous communication, with the choice between IPv4 and IPv6 addresses, with multiple local network interfaces being available, and more. With the development of new transport protocols, such as QUIC, an application may even be able to choose a different transport protocol than TCP, which fundamentally offers the same service. It would be beneficial to develop a future networking API with a better abstraction, so applications can make use of the different options without having to implement a complex decision logic themselves. The Transport Services (TAPS) Working Group in the IETF is currently developing such an abstraction, which will be proposed as a new standard networking API. It will be fundamentally event-based and object-oriented, presenting a simple Connection and Message abstraction to applications. An application only has to specify abstract requirements for a new Connection, such as reliable and in-order Delivery of messages, and the Transport System below the API will choose a transport protocol and network configuration. This talk presents the current TAPS architecture and API elements, and calls for the Linux community to get involved, for example by developing a new implementation of this API or reviewing and commenting on the documents.

uk: вхід без авторизації


The State of Your Supply Chain

uk: Комп'ютерні науки

description: Container security often focuses on runtime best-practices whilst neglecting delivery of the software in the supply chain. Application, library, and OS vulnerabilities are a likely route to data exfiltration, and emerging technologies in the container ecosystem offer a new opportunity to mitigate this risk. Treating containers as immutable artefacts and injecting configuration allows us to "upgrade" images by rebuilding and shipping whole software bundles, avoiding configuration drift and state inconsistencies. This makes it possible to constantly patch software, and to easily enforce governance of artefacts both pre- and post-deployment. In this talk we detail an ideal, security-hardened container supply chain, describe the current state of the ecosystem, and dig into specific tools. Grafeas, Kritis, in-toto, Clair, Micro Scanner, TUF, and Notary are covered, and we demo how to gate container image pipelines and deployments on cryptographically verified supply chain metadata.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: Thunderbolt 3 is a high-speed IO technology that can be used to connect docks, graphic cards or other peripherals requiring high speed. However, the mechanism that allows these high speeds also poses a security risk because malicious devices could obtain sensitive information from the computer's memory. As a result kernel provides an interface to authorize thunderbolt devices before they can be used. The talk will explain the technology and explain the enablement we did to make thunderbolt 3 work for GNU/Linux. Thunderbolt 3 is a relatively new technology to connect peripherals to a computer. Because it can access the computer's resources directly, it allows for very high speeds: it is fast enough to drive external graphics cards. However, the mechanism that allows these high speeds also poses a security risk because malicious devices could obtain sensitive information from the computer's memory. Version 3 of the Thunderbolt interface therefore provides security levels in order to mitigate the aforementioned security risk that connected devices pose to the system. As a result, devices need to be authorized manually. The talk aims to provide an overview of the Thunderbolt technology and will try to clarify some of the confusing aspects, e.g. the many modes and features of the USB type C connector that Thunderbolt 3 uses. Finally, the talk will show how some tricky user experience problems were solved,

uk: вхід без авторизації

name: Thunderbolt 3 hardware enablement for GNU/Linux


uk: Комп'ютерні науки

description: Titus is a multitenant scheduler that runs a variety of workloads that vary from online workloads which serve customer traffic to big data workloads which perform machine learning. Getting all of these workloads to cooperate on a shared pool of resources together. Just to add a bit of complexity to the mix, these workloads all run on the cloud, and a shared storage and network fabric. Come to this talk to learn about how our approach to multi-tenancy works, as well as some of the challenges we faced along the way. Titus is a system which allows users to submit arbitrary container workloads to the cloud, and get their workloads running across many thousands of cores or more. This comes with a variety of challenges. We attacked this problem with a three-pronged approach. Our approach to scheduling is multi-tenant first. Our scheduler understands different workloads and the fact that different workloads have different Service Level Objectives. In addition to this, it understands the cloud, and the fact it’s a shared control plane. Lastly, we’ve had to teach our scheduler to handling situations during failover, and when scaling up is key versus traditional scheduling. Our approach to systems is evolving. Historically, our fleet was many single-tenant VMs. We’ve attacked systems level multi-tenancy from the multiple perspectives. The first of these involved giving our user the APIs that were as close to what they had on the VM. Subsequently, we’ve tried to enable security mechanisms like seccomp and apparmor that allow us to run nearly any workload on Titus. Lastly, we’re still figuring out resource isolation. Cgroups have come a long way, but there is a long way to go ahead before we can be as good as VMs. All of our infrastructure runs on the cloud. We decided that our approach to scheduling, and systems multi-tenancy should be cloud native, and leverage as many mechanisms as possible that already exist in the cloud rather than invent our own. Although this gave us a massive head start, it didn’t come for free. We had to solve problems like coordination-free optimistic interactions with our SDN, and solutions to shared-storage.

uk: вхід без авторизації

name: Titus: Adventures in Multi-tenant Scheduling


uk: Комп'ютерні науки

description: We’ll look at patterns and anti-patterns for self-contained, immutable runtime environment for applications using Habitat, with a focus on special cases, integrated testing and advanced hacks.

uk: вхід без авторизації

name: To Run an App With Guarantees We Must First Create The Universe: Lessons from managing self-contained application runtimes in production


uk: Комп'ютерні науки

description: I’d like to show you how to find bugs in Linux systems using machine learning, when paired with the totally seemingly useless and annoying false positives that come out of your integration tests. We’ve all been frustrated unreproducible bugs in Linux … And also by stupid test-flakes that show up as failures in integration tests even though nothing related has changed. Both unproducible bugs and test flakes are the result of permutations in timing, load, ordering. I want to prove that unreproducible bugs are the same thing as test flakes. And we have a massive source of data here, millions of records of test flakes. Lets take a look at how the Cockpit project trains bots to correlate these flakes into unsupervised clusters. and automatically make use of the data, identifying real bugs, or simply retriggering tests. We’ll dive into details about Normalized Compression Distance, Unsupervised Clustering, TF-IDF and many other simple techniques used to zero in on the bugs.

uk: вхід без авторизації

name: Using Machine Learning to find Linux bugs


Using systemd to high level languages

uk: Комп'ютерні науки

description: There is so much more than you can so, than just starting and stooping your service, when you start to interact with systemd from within your application. Lets find out!!! We'll explore how can you interact with systemd from your aplication, by describing how pystemd, a python binding for systemd, was created at facebook, and what type things your programs can take advantage if you just interact with systemd more. From executing section of your code securely to create development environment on the fly, There is so much more than you can so, than just starting and stooping your service, when you start to interact with systemd from within your application.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: TL;DR: Stop making "Desktop Linux" a moving target by agreeing on a minimal baseline that third-party application can take for granted to exist on each Desktop Linux system. To make "Desktop Linux" a viable platform (as in: Windows, macOS) to develop against, we need to insist on backward compatibility and either need to find the "least common denominator" by experimentation (as we have done for over a decade now), or get the main Linux distributions (Ubuntu, Fedora, openSUSE, Debian, etc.) to agree on a certain set of infrastructure that can be expected to be "there", in a consistant way, without unnecessary differences. A guaranteed minimal set of infrastructure (available in the default installation of all distributions) with guaranteed backward compatibility is required for "Desktop Linux" to become viable as a platform.

uk: вхід без авторизації

name: 2018 Desktop Linux Platform Issues: Will the year of the Linux desktop ever arrive? What can we do to make it happen?


A debugger from scratch

uk: Комп'ютерні науки

description: At some stage in your programming life you may well have used a debugger, but did you wonder how it was able to step into and control your executable? In this talk we'll see how debuggers work by building one from scratch in a few lines of Go. In this talk Liz will explore how a debugger gains control of a process with the all-powerful ptrace system call. You'll see how we can find the machine code that corresponds to a line of human-readable source code and vice-versa. She will show how breakpoints are set, explain how the stack gets built up, and demonstrate how to generate a stack trace showing the path to that breakpoint. Even if you know nothing about machine code, you'll leave this talk with a better understanding of how a computer runs an executable, and how a debugger is able to start and stop the executable as you wish.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: Open Container Initiative (OCI) started in 2015 to make different implementations of container runtimes and images compliant with well-defined specifications. Together with other folks at Kinvolk, I have been involved in various OCI projects since months, and encountered various issues that occur in runtime specs and runtime-tools for verification. Since we live in a real world, not everything works well as expected. I’m going to talk about practical issues, and possible ways to get it improved. Open Container Initiative (OCI) defines container runtime specs (https://github.com/opencontainers/runtime-spec) as well as container image specs (https://github.com/opencontainers/image-spec) and distribution spec (https://github.com/opencontainers/distribution-spec). There is also runtime-tools (https://github.com/opencontainers/runtime-tools) that helps container runtime to verify compliance of the runtime specifications. The standard container runtime is runc (https://github.com/opencontainers/runc) that is included in multiple high-level container managers like Docker or containerd. Most of the practical issues arise when specification is not clearly defined in the first place, or when container runtimes have own reasons for not being compliant with the specs, or when there’s no consensus in the community how it should proceed. On the other hand, container orchestration systems like Kubernetes have defined their own interfaces such as Container Runtime Interface (CRI). The different interfaces (OCI runtime and CRI) exist at different layers in the software stack. I'll show how CRI depends on OCI and some mismatches between them. In this talk I want to introduce such practical issues, and try to suggest how we should proceed regarding spec compliance.

uk: вхід без авторизації

name: Being compliant with Open Container Initiative Spec


uk: Комп'ютерні науки

description: To bring cool ideas to life, the Linux kernel and user space need to work together. The core kernel is a stable ABI base, a common denominator for everything that builds on top. In contrast, BPF is a specific know-how, a secret sauce of the cool idea. The Linux kernel needs BPF to stay relevant, and BPF has to become friendlier to programmers. This talk explores the steps taken towards this long-term goal, from recently introduced BPF-to-BPF functions and type information to bounded loops, memory allocation, and beyond.

uk: вхід без авторизації

name: BPF and the future of the kernel extensibility


uk: Комп'ютерні науки

description: How Facebook uses Chef to manage many different largescale heterogenous environments. Facebook uses chef to deploy and manage a wide array of environments across our fleet, ranging from network switches to mobile phones to LXC containers to off-network devices. This talk will explore our toolchain for building, testing, and running Chef code in this diverse ecosystem of environments and includes stories about it's evolution and development. Zeal Jagannatha

uk: вхід без авторизації

name: Chef in Strange Places: Chef-Solo at Facebook


uk: Комп'ютерні науки

description: A fundamental query in labeled graphs is to determine if there exists a path between a given source and target vertices, such that the path satisfies a given label constraint. One of the powerful forms of specifying label constraints is through regular expressions, and the resulting problem of reachability queries under regular simple paths (RSP) form the core of many practical graph query languages such as SPARQL from W3C, Cypher of Neo4J, Oracle's PGQL and LDBC's G-CORE. Despite its importance, since it is known that answering RSP queries is NP-Hard, there are no scalable and practical solutions for answering reachability with full-range of regular expressions as constraints. In this paper, we circumvent this computational bottleneck by designing a random-walk based sampling algorithm called ARRIVAL, which is backed by theoretical guarantees on its expected quality. Extensive experiments on billion-sized real graph datasets with thousands of labels show that ARRIVAL to be 100 times faster than baseline strategies with an average accuracy of 95%.

uk: вхід без авторизації

name: Efficiently Answering Regular Simple Path Queries on Large Labeled Networks


Eine Stadt wird zum Verkehrslabor

name: Anwendungsplattform Intelligente Mobilität (AIM)

description: Die Großforschungsanlage AIM (Anwendungsplattform Intelligente Mobilität) beim Deutschen Zentrum für Luft- und Raumfahrt (DLR) in Braunschweig steht Forschern von nun an für ihre Projekte zur Verfügung. Durch AIM wird eine Stadt, mit all den Wegen, die ihre Bewohner im Alltag zurücklegen, zum Verkehrslabor. Die niedersächsische Ministerin für Wissenschaft und Kultur, Dr. Gabriele Heinen-Kljajić, gab bei einem offiziellen Auftakt am 17. Juli 2014 den Startschuss. Verkehrsforscher des DLR, anderer Forschungsorganisationen und Unternehmen können mit AIM neue Technologien für sichere Fahrerassistenz sowie ein modernes und effizientes Verkehrsmanagement entwickeln und erproben. Den Forschern stehen eine Forschungskreuzung, eine Referenzstrecke, zur Beobachtung des Fahrerverhaltens ausgerüstete Fahrzeuge, verschiedene Verkehrssimulatoren und weitere Anlagen zur Verfügung. "AIM ist in Deutschland ein wichtiges Instrument zur Entwicklung von Fahrerassistenzsystemen.

uk: вхід без авторизації

name: Deutsches Zentrum für Luft- und Raumfahrt (DLR)

addressLocality: Cologne

addressRegion: North Rhine-Westphalia


description: Der Lander Philae hat am 12. November 2014 auf der Kometenoberfläche Churyumov-Gerasimenko aufgesetzt. Damit steht erstmals ein vom Menschen geschaffenes Gerät auf einem Kometen und misst unmittelbar vor Ort Daten. "Philae spricht zu uns - wir sind auf der Kometenoberfläche", sagt Lander-Projektleiter Dr. Stephan Ulamec vom Deutschen Zentrum für Luft- und Raumfahrt (DLR). Um 17:09 Uhr jubelte das Team im Lander Control Center (LCC) des DLR, das den Lander steuert und betreibt. ---------------------------------------------------------------------------

uk: вхід без авторизації

name: #6 Aufzeichnung: Livestream der Kometenlandung vom 12.11.2014


description: Der Lander Philae hat am 12. November 2014 auf der Kometenoberfläche Churyumov-Gerasimenko aufgesetzt. Damit steht erstmals ein vom Menschen geschaffenes Gerät auf einem Kometen und misst unmittelbar vor Ort Daten. "Philae spricht zu uns - wir sind auf der Kometenoberfläche", sagt Lander-Projektleiter Dr. Stephan Ulamec vom Deutschen Zentrum für Luft- und Raumfahrt (DLR). Um 17:09 Uhr jubelte das Team im Lander Control Center (LCC) des DLR, das den Lander steuert und betreibt.

uk: вхід без авторизації

name: #3 Aufzeichnung: Livestream der Kometenlandung vom 12.11.2014


uk: вхід без авторизації

name: #4 Aufzeichnung: Livestream der Kometenlandung vom 12.11.2014


description: Der Lander Philae hat am 12. November 2014 auf der Kometenoberfläche Churyumov-Gerasimenko aufgesetzt. Damit steht erstmals ein vom Menschen geschaffenes Gerät auf einem Kometen und misst unmittelbar vor Ort Daten. "Philae spricht zu uns - wir sind auf der Kometenoberfläche", sagt Lander-Projektleiter Dr. Stephan Ulamec vom Deutschen Zentrum für Luft- und Raumfahrt (DLR). Um 17:09 Uhr jubelte das Team im Lander Control Center (LCC) des DLR, das den Lander steuert und betreibt. --------------------------------------------------------------------------

uk: вхід без авторизації

name: #5 Aufzeichnung: Livestream der Kometenlandung vom 12.11.2014


uk: вхід без авторизації

name: #1 Aufzeichnung: Livestream der Kometenlandung vom 12.11.2014


uk: вхід без авторизації

name: #2 Aufzeichnung: Livestream der Kometenlandung vom 12.11.2014


uk: Комп'ютерні науки

description: Not all projects can be greenfield. As developers we are often handed legacy code that we need to extend and improve upon. The BBC is no different. I am going to discuss the methods that the BBC are using to migrate their legacy products from monolithic to microservice based architectures, with examples of how BBC Knowledge and Learning have enhanced the user experience whilst never once disrupting service.

uk: вхід без авторизації

name: Keynote: Dimantling Monoliths: using microservices and continuous delivery at the BBC


Keynote: The npm workflow

uk: Комп'ютерні науки

description: Lots of people use npm every day but are only just scratching the surface of what it can do. npm isn't just for installing packages, it's a way of removing friction from every step of your development workflow. In this talk we'll cover how to use npm to save you and your team time and effort. We'll start at the most basic steps and go through to advanced techniques that will surprise even pro users. In particular we'll talk about using npm effectively as a front-end developer, npm's biggest and fastest-growing group of users.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: Angular 2 kommt … und es macht alles anders! Ist das wirklich so? Was hat das für meine AngularJS-Anwendung zu bedeuten? Was kann ich tun, um mich heute schon auf einen sanften Umstieg vorzubereiten? Ich habe verschiedene Möglichkeiten der Migration von AngularJS auf Angular 2 in Projekten ausprobiert und teile meine Erfahrungen mit euch.

uk: вхід без авторизації

name: Praxisbericht: AngularJS Projekte sicher auf Angular 2 migrieren


Reactive Programming

uk: Комп'ютерні науки

description: Reactive Programming mit JavaScript macht Spaß – und zwar nicht nur den Entwicklern, sondern auch den Benutzern einer Applikation. Das Ziel dieses Paradigmas ist eine Anwendung, die den Benutzer im Fokus hat, ihn kontinuierlich informiert hält und auch unter suboptimalen Bedingungen noch eine beeindruckende User Experience vermittelt. Die gute Nachricht hinter all dem: Sämtliche Technologien, die für ein reaktives System benötigt werden, stehen schon zur Verfügung. JavaScript-Entwickler müssen noch nicht einmal etwas Neues lernen, sondern lediglich ihre Vorgehensweise bei der Umsetzung etwas anpassen und ein paar Regeln folgen.

uk: вхід без авторизації


On Connection Lost

uk: Комп'ютерні науки

description: Das war die Anforderung unseres Kunden: Die Mitarbeiter werden mit der Applikation unterwegs sein. Sie wollen dann Dokumente lesen, Ergebnisse erfassen und Bilder hochladen. Wenn sie zurückkommen, erwarten sie, dass sich die Änderungen automatisch synchronisiert haben. Die HTML-5-Technologien AppCache, IndexedDB und File API geben uns die technischen Möglichkeiten. Sowohl der Applikationscode als auch die Daten können im Browser lokal gespeichert werden. Allerdings verlangt uns eine solche Applikation ab, einige vertraute Muster zu überdenken: Wer ist der Herr über die Daten? Der Server oder der Client? Wie gehen wir mit Updates der Applikation um? Welche Daten müssen wir herunterladen, bevor jemand offline geht? Wie können wir die Offlinefunktionen automatisiert testen? Und was bedeutet offline eigentlich genau? Ich will Ihnen zeigen, was wir diskutiert haben und wie wir diese Fragen für uns beantwortet haben.

uk: вхід без авторизації


Angular 2: Change Detection explained

uk: Комп'ютерні науки

description: Was ist Change Detection? Was löst Veränderungen aus? Und was genau ist eine Veränderung in dem Zusammenhang überhaupt? In diesem Talk wollen wir einen Blick auf einen der wichtigsten Teile des Angular-2-Frameworks werfen. Wir werden lernen, wie Angular Zones und die unterschiedlichen Change-Detection-Implementierungen nutzt, um unsere Anwendungen schnell und responsive zu machen.

uk: вхід без авторизації


JavaScript for Cyborgs

uk: Комп'ютерні науки

description: Cyborgs are all around us, and they're becoming self-aware. This talk uses NFC as a demonstration of opportunities and pitfalls of open hardware for the human body. Our live-coding journey will take us through an intro to NFC protocols by building an app for the NFC transponder implanted into my hand.

uk: вхід без авторизації


uk: Комп'ютерні науки

description: AngularJS ist eines der bekanntesten JavaScript-Frameworks. Die neue Version, Angular 2, befindet sich zurzeit in der Beta-Phase. Sie baut auf den Erfahrungen und positiven Eigenschaften von Angular 1 auf und soll schneller und moderner sein. Doch man braucht kein Vorwissen in Angular, um Angular 2 schon heute zu verwenden. Das Framework wurde komplett neu geschrieben, ist konzeptionell einfacher und verwendet eine neue Syntax. Dieser Talk ist eine Einführung in das Framework: wir lernen Components kennen, werfen einen Blick auf die Template-Syntax, Pipes und Directives und sehen uns Dependency Injection an.

uk: вхід без авторизації

name: Erste Schritte mit Angular 2 - Components; Directives, Templates