December 14, 2024
Atomfs presented at OCI weekly discussion
Serge Hallyn
the original docker format. Each container image is represented as an
array of ‘layers’, each of which is a .tar.gz. To unpack the container
image, untar the first, then untar the second on top of the first, etc.
Several years ago, while we were working on a product which ships its
root filesystem (and of course containers) as OCI layers, Tycho Andersen
(https://tycho.pizza/) came up with the idea of ‘atomfs’ as a way to
avoid some of the deficiencies of tar
(https://www.cyphar.com/blog/post/20190121-ociv2-images-i-tar). In
‘atomfs’, the .tar.gz layers are replaced by squashfs (now optionally
erofs) filesystems with dm-verity root hashes specified. Mounting an
image now consists of mounting each squashfs, then merging them with
overlay. Since we have the dmverity root hash, we can ensure that the
filesystem has not been corrupted without having to checksum the files
before mounting, and there is no tar unpacking step.
This past week, Ram Chinchani presented atomfs at the OCI weekly
discussion, which you can see here
https://www.youtube.com/watch?v=CUyH319O9hM starting at about 28
minutes. He showed a full use cycle, starting with a Dockerfile,
building atomfs images using stacker, mounting them using atomfs, and
then executing a container with lxc. Ram mentioned his goal is to have
a containerd snapshotter for atomfs soon. I’m excited to hear that, as
it will make it far easier to integrate into e.g. kubernetes.
Exciting stuff!
December 13, 2024
How does OpenSearch work?
Ubuntu Blog
OpenSearch is an open source search and analytics suite that developers use to build solutions for search, data observability, data ingestion, security information and event management (SIEM), vector database, and more. It is designed for scalability, offering powerful full-text search capabilities and supporting various data types, including structured and unstructured data. OpenSearch has rapidly developed into a standalone platform with unique features and capabilities.
OpenSearch is Apache 2.0 licensed software, and is run, maintained and advanced by the community. OpenSearch includes a network of partners and is open to contribution. The organization believes that great open source software is built together with a diverse community of contributors. The mission of the OpenSearch Software Foundation is to provide infrastructure and other resources to enable the long-term sustainability of the OpenSearch open source project and the OpenSearch ecosystem.
In this blog, we’ll explore how OpenSearch works and what it’s for, examining its key use cases and components.
OpenSearch enterprise use cases
Search | OpenSearch enhances your website, data lake catalog, or e-commerce search capabilities with full-text querying, autocomplete, scroll search, and customizable scoring and ranking. |
Analytics and machine learning | You can use OpenSearch in multiple analytics solutions such as events analytics, trace analytics, and machine learning, which uses algorithms such as anomaly detection and data clustering. |
Security | Security information and event management (SIEM) solutions can use OpenSearch to investigate, detect, analyze, and respond to security threats that can jeopardize the success of organizations and their online operations. |
Observability | You can use OpenSearch to create observability applications through the OpenSearch Dashboard. You can also use it to schedule, export, and share reports. |
How does OpenSearch work?
OpenSearch consists of a data store and search engine called OpenSearch, and a visualization and user interface called OpenSearch Dashboards. In addition, users can extend the functionality of OpenSearch with a selection of plugins that enhance search, security, performance analysis, machine learning, and more.
Search engine and data store
OpenSearch features a distributed design that allows users and applications to interact with clusters. Each cluster consists of one or more nodes running on servers that store data and handle search requests. Similar to how databases and tables are organized in relational databases, OpenSearch uses indices to structure its data. The data within a cluster is organized by mapping each index to a primary shard, which is then replicated to one or more replica shards. This setup not only safeguards your data against hardware failures but also increases capacity for handling read requests.
The diagram below illustrates an example of an OpenSearch cluster, displaying OpenSearch nodes, OpenSearch Dashboard, and data sources.
End users can interact directly with the OpenSearch Dashboards, for example to perform data analysis tasks in order to improve business processes. However, before users can access the Dashboard, data sources need to be ingested into the OpenSearch cluster. This data source can be in different formats like log files, metrics, JSON documents, etc.
A cluster can contain various types of nodes: main, coordinating and data nodes. Each node has a different role:
- Cluster managers – Manage the overall operation of a cluster and keep track of the cluster state. This includes creating and deleting indexes, keeping track of the nodes that join and leave the cluster, checking the health of each node in the cluster (by running ping requests), and allocating shards to nodes.
- Data nodes – Store and search data. These nodes perform all data-related operations (indexing, searching, aggregating) on local shards. These are the worker nodes of a cluster and need more disk space than any other node type.
- Coordinating nodes – Delegate client requests to shards on the data nodes, collect and aggregate the results into one final result, and send this result back to the client. Coordinating nodes manage outside requests like the OpenSearch Dashboard and other client libraries.
Visualization and user interface
As described above, OpenSearch Dashboards is included in the OpenSearch ecosystem, however, it is optional for users. OpenSearch Dashboards is an open-source, integrated visualization tool that allows users to explore their data in OpenSearch. From real-time application monitoring, threat detection, and incident management to personalized search, OpenSearch Dashboards represent trends, outliers, and patterns in data graphically. The image below shows a sample of data visualizations in OpenSearch Dashboards.
Figure 1: OpenSearch Dashboards weblogs (source: Sample dashboard in AWS)
Other features and plug-ins
OpenSearch provides several features to help index, secure, monitor, and analyze your data. Most OpenSearch plugins have associated OpenSearch Dashboards plugins that provide a convenient, unified user interface.
- Anomaly detection – Identify atypical data and receive automatic notifications.
- SQL – Use SQL or a Piped Processing Language (PPL) to query your data.
- Index State Management – Automate index operations.
- Search methods – From traditional lexical search to advanced vector and hybrid search, discover the optimal search method for your use case.
- Machine learning – Integrate machine learning models into your workloads.
- Workflow automation – Automate complex OpenSearch setup and preprocessing tasks.
- Performance evaluation – Monitor and optimize your cluster.
- Asynchronous search – Run search requests in the background.
- Cross-cluster replication – Replicate your data across multiple OpenSearch clusters.
Enterprise OpenSearch solution
Charmed OpenSearch builds on the OpenSearch upstream by integrating automation to streamline production clusters’ deployment, management, and orchestration. The operator enhances efficiency, consistency, and security. Its rich features include high availability, seamless scaling features for deployments of all sizes, both http and data-in-transit encryption, multi-cloud support, safe upgrades without downtime, roles and plugin management and data visualization through Charmed OpenSearch Dashboards. Charmed OpenSearch provides a comprehensive solution that simplifies complex operations, supports scalable infrastructure, and ensures high performance and security.
With the Charmed OpenSearch operator, you can deploy and run OpenSearch on physical and virtual machines (VM) and other cloud and cloud-like environments, including AWS, Azure, Google Cloud, OpenStack, and VMware. On top of this, Charmed OpenSearch guarantees security patching for critical and high-severity Common Vulnerabilities and Exposures (CVEs) with ten years of security maintenance and 24/7 access to a world-class support team.
Get started with Charmed OpenSearch by watching this Getting Started Webinar and following this tutorial.
Conclusion
OpenSearch offers a robust search service, effective data storage, and impressive visualization features, allowing it to effectively address various use cases such as application search, log analytics, data observability, and data ingestion. Additionally, its architecture is designed to ensure optimized search and analytics capabilities. OpenSearch is gaining significant traction due to its open source license and rapid innovations over the past few years.
To learn more, visit canonical.com/data/opensearch/what-is-opensearch
Explore more or contact our team for your OpenSearch needs.
Reference: www.opensearch.org
What is RAG?
Ubuntu Blog
In a 2020 paper, Patrick Lewis and his research team introduced the term RAG, or retrieval-augmented generation. This technique enhances generative AI models by utilizing external knowledge sources such as documents and extensive databases. RAG addresses a gap in traditional Large Language Models (LLMs). While traditional models rely on static knowledge already contained within them, RAG incorporates current information that serves as a reliable source of truth for LLMs. Although LLMs can rapidly understand and respond to prompts, without RAG, they often fall short in providing current or more specific information.
One practical application of RAG is in conversational agents and chatbots. RAG models enhance these systems by enabling them to fetch contextually relevant information from external sources. This capability ensures that customer service chatbots, virtual assistants, and other conversational interfaces deliver accurate and informative responses during interactions. Another use case is in advanced question-and-answer systems, where RAG models help individuals obtain answers to queries, such as support ticket responses. Additionally, RAG can be utilized in content recommendation systems, offering personalized recommendations by retrieving relevant information to enhance users’ experiences and content engagement.
Benefits of using RAG
Data sources that build trust
When using a machine learning model for enterprise applications, such as chatbots or for searching sensitive data, RAG provides the models with verifiable information that can be cited. This approach enables the models to focus on less ambiguous contexts, significantly reducing the likelihood of generating incorrect outputs – a phenomenon often referred to as “hallucination”.
Up-to-date information
With RAG, you can load current and reliable data, ensuring that your model will retrieve accurate information.
Simplicity of RAG
Lewis and three coauthors published a paper titled “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” in which they demonstrated RAG with just five lines of code. This implementation resulted in a model that includes a question encoder, a retriever, and a generator for context-aware answers. The simplicity of this implementation showcases that RAG is relatively easy as a concept and project. However, complexity does increase in production and large-scale deployments.
Reduced costs of continuous model re-training
Continuously re-training machine learning models can be expensive. By utilizing RAG, there is no need to depend on time-intensive and costly parameter retraining. This has the potential to lower both computational and financial costs associated with running LLM-powered chatbots in an enterprise setting.
How RAG works
When a query is made in an AI chatbot, the RAG-based system first retrieves relevant information from a large dataset or knowledge base, then this information is used to inform and guide the generation of the response. The RAG-based system consists of two key components. The first component is the Retriever, which is responsible for locating relevant pieces of information that can help answer a user query. It searches a database to select the most pertinent information. This information is then provided to the second component, the Generator. The Generator is a large language model that produces the final output. See the figure below:
Before using your RAG-based system, you must first create your knowledge base, which consists of external data that is not included in your LLM training data. This external data can originate from various sources, including documents, databases, and API calls. Most RAG systems utilize an AI technique called model embedding, which converts data into numerical representations and stores it in a vector database. By using an embedding model, you can create a knowledge model that is easily understandable and readily retrievable in the context of AI. Once you have a knowledge base and a vector database set up, you can now perform your RAG process; here is a conceptual flow:
The RAG-based system has five straightforward steps:
- You can begin by entering your prompt or query.
- Retriever is used for searching for relevant information from a knowledge base. The relevance can be determined using mathematical vector calculations and representations through a vector search and database functionality.
- The relevant information to provide enhanced context is retrieved and passed to the generator.
- Query and prompts are now enriched with this context and are ready to be augmented for use with a large language model using prompt engineering techniques (LLM). The augmented prompt enables the language model to respond accurately to your query.
- Finally, the generated text response is delivered to you.
Canonical for your RAG requirements
Build the right RAG architecture with Canonical RAG workshop
Canonical offers a 5-day workshop designed to help you start building your enterprise RAG systems. By the end of the workshop, you will have a thorough understanding of RAG and LLM theory, architecture, and best practices. Together, we will develop and deploy solutions tailored to your specific needs.
Explore more and contact our team for your RAG needs.
December 12, 2024
E328 Reunião De Pies
Podcast Ubuntu Portugal
Continuam as discussões sobre auto-alojamento («self-hosting»), onde recebemos sugestões e opiniões de ouvintes que exploraram esse tema; o Miguel continua a não poder usar um VPN no telefone; abordámos o roteiro de lançamento da próxima versão de Ubuntu, Plucky Puffin e o próximo encontro da Comunidade em Sintra; demos as boas-vindas a novos Snaps criados pela Comunidade e babámos um bocadinho com novos brinquedos da gama Raspberry Pi! E no fim, os patronos tiveram direito a um teatro de fantoches com a República.
Já sabem: oiçam, subscrevam e partilhem!
- RPi CM5: https://9to5linux.com/raspberry-pi-compute-module-5-launches-as-a-modular-version-of-raspberry-pi-5
- Rpi 500 + Monitor: https://9to5linux.com/raspberry-pi-500-computer-launches-with-the-official-raspberry-pi-monitor
- Jeff Geerling testa um Pi500: https://youtu.be/5YfJWYELA3k
- Leepsvideo testa um Pi500: https://youtu.be/JY_Zv7DCFrI
- Jeff Geerling descasca um Pi500: https://youtu.be/omYWRb1dLA4
- Jogo Capitals em Snap: https://snapcraft.io/capitals
- Jogo Capitals no Github: https://github.com/lapisdecor/capitals
- LoCo PT: https://loco.ubuntu.com/teams/ubuntu-pt/
- Nitrokey: https://shop.nitrokey.com/shop?aff_ref=3
- Mastodon: https://masto.pt/@pup
- Youtube: https://youtube.com/PodcastUbuntuPortugal
Apoios
Podem apoiar o podcast usando os links de afiliados do Humble Bundle, porque ao usarem esses links para fazer uma compra, uma parte do valor que pagam reverte a favor do Podcast Ubuntu Portugal. E podem obter tudo isso com 15 dólares ou diferentes partes dependendo de pagarem 1, ou 8. Achamos que isto vale bem mais do que 15 dólares, pelo que se puderem paguem mais um pouco mais visto que têm a opção de pagar o quanto quiserem. Se estiverem interessados em outros bundles não listados nas notas usem o link https://www.humblebundle.com/?partner=PUP e vão estar também a apoiar-nos.
Atribuição e licenças
Este episódio foi produzido por Diogo Constantino, Miguel e Tiago Carrondo e editado pelo Senhor Podcast. O website é produzido por Tiago Carrondo e o código aberto está licenciado nos termos da Licença MIT. (https://creativecommons.org/licenses/by/4.0/). A música do genérico é: “Won’t see it comin’ (Feat Aequality & N’sorte d’autruche)”, por Alpha Hydrae e está licenciada nos termos da CC0 1.0 Universal License. Este episódio e a imagem utilizada estão licenciados nos termos da licença: Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0), cujo texto integral pode ser lido aqui. Estamos abertos a licenciar para permitir outros tipos de utilização, contactem-nos para validação e autorização.
December 11, 2024
December 09, 2024
Ubuntu Weekly Newsletter Issue 869
The Fridge
Welcome to the Ubuntu Weekly Newsletter, Issue 869 for the week of December 1 – 7, 2024. The full version of this issue is available here.
In this issue we cover:
- Ubuntu Forums Migration
- Ubuntu Foundations 25.04 – Plucky Puffin Roadmap
- Ubuntu Membership Board 2024 elections are open!
- 2024 Ubuntu Technical Board – Voting Cycle Extension
- Welcome New Members and Developers
- Ubuntu Stats
- Hot in Support
- LXD: Weekly news #374
- Rocks Public Journal; 2024-12-06
- Other Meeting Reports
- Upcoming Meetings and Events
- Ubuntu Portugal: November 2024 activity report
- UbuCon Asia 2025 (Aug 30-31) is now looking for sponsors
- LoCo Events
- LXD: Time to pick a snap channel
- Landscape: Feedback on the new web portal
- Call for testing: ubuntu-frame, mir-test-tools (Mir 2.19 update)
- PSA: updating the default tracks to `24` (`ubuntu-frame*`, `*-test-tools`)
- Introducing Launchpad Bug Templates
- Documentation, development and design for technical authors
- Other Community News
- Canonical News
- In the Blogosphere
- Other Articles of Interest
- Featured Audio and Video
- Updates and Security for Ubuntu 20.04, 22.04, 24.04, and 24.10
- And much more!
The Ubuntu Weekly Newsletter is brought to you by:
- Krytarik Raido
- Bashing-om
- Chris Guiver
- Wild Man
- Din Mušić
- Cristovao Cordeiro – cjdc
- And many others
If you have a story idea for the Weekly Newsletter, join the Ubuntu News Team mailing list and submit it. Ideas can also be added to the wiki!
December 08, 2024
Ubuntu Forums Migration
The Fridge
The Support and Help Section of Ubuntu Discourse is now officially alive and open for business.
If you wish to start a support thread, please click on this link. The first two links lead to wiki pages. Or, click on whichever of the other links is most appropriate for your issue, and a new page will open with a “Create a new Topic” message editor already open for you.
Other useful links to Ubuntu Discourse are:
- Ubuntu Discourse home page – https://discourse.ubuntu.com/
- Ubuntu Discourse Support and Help section – https://discourse.ubuntu.com/c/support-and-help/306
You are reminded that it will no longer be possible to start new threads on this old site with effect from December 9th 2024 or thereabouts. However, forum members will still be able to post to existing threads for a short period so that those starting threads just before the transition are not left without support. Thereafter, this site will become read only on January 9th 2025.
A special thanks to all Staff (past and present) which include
- ajgreeny – The Ubuntu Forum Community
- Bashing-om – Weekly Newsletter, Sub-Forum Moderator
- CharlesA – The Ubuntu Forum Community
- coffeecat – The Ubuntu Forum Community, Administrator
- deadflowr – The Ubuntu Forum Community
- DuckHook – The Ubuntu Forum Community, Super Moderator
- Frogs Hair – The Ubuntu Forum Community
- howefield – The Ubuntu Forum Community, Administrator
- Irihapeti – The Ubuntu Forum Community
- jeremy31 – The Ubuntu Forum Community
- oldfred – The Ubuntu Forum Community, Super Moderator
- QIII – The Ubuntu Forum Community, Administrator
- slickymaster – The Ubuntu Forum Community
- wgarcia – The Ubuntu Forum Community
- wildmanne39 – The Ubuntu Forum Community, Super Moderator
- and all past staff.
Thank you all.
Posted on Thu Nov 21 2024 by Ubuntu Forums Staff.
December 06, 2024
The new Firebuild release contains plenty of small fixes and a few notable improvements.
Experimental macOS support
The most frequently asked question from people getting to know Firebuild was if it worked on their Mac and the answer sadly used to be that well, it did, but only in a Linux VM. This was far from what they were looking for. 🙁
Linux and macOS have common UNIX roots, but porting Firebuild to macOS included bigger challenges, like ensuring that dyld(1)
, macOS’s dynamic loader initializes the preloaded interceptor library early enough to catch all interesting calls, and avoid using anything that uses malloc()
or thread local variables which are not yet set up then.
Preloading libraries on Linux is really easy, running LD_PRELOAD=my_lib.so ls
just works if the library exports the symbols to be interposed, while macOS employs multiple lines of defense to prevent applications from using unknown libraries. Firebuild’s guide for making DYLD_INSERT_LIBRARIES honored on Macs can be helpful with other projects as well that rely on injecting libraries.
Since GitHub’s Arm64 macOS runners don’t allow intercepting binaries with arm64e ABI yet, Firebuild’s Apple Silicon tests are run at Bitrise, who are proud to be first to provide the latest Xcode stacks and were also quick to make the needed changes to their infrastructure to support Firebuild (thanks! ).
Firebuild on macOS can already accelerate simple projects and rebuild itself with Xcode. Since Xcode introduces a lot of nondeterminism to the build, Firebuild can’t shine in acceleration with Xcode yet, but can provide nice reports to show which part of the build is the most time consuming and how each sub-command is called.
If you would like to try Firebuild on macOS please compile it from the GitHub repository for now. Precompiled binaries will be distributed on the Mac App Store and via CI providers. Contact us to get notified when those channels become available.
Dealing with the ‘Epochalypse’
Glibc’s API provides many functions with time parameters and some of those functions are intercepted by Firebuild. Time parameters used to be passed as 32-bit values on 32-bit systems, preventing them to accurately represent timestamps after year 2038, which is known as the Y2038 problem or the Epochalypse.
To deal with the problem glibc 2.34 started providing new function symbol variants with 64-bit time parameters, e.g clock_gettime64()
in addition to clock_gettime()
. The new 64-bit variants are used when compiling consumers of the API with _TIME_BITS=64
defined.
Processes intercepted by Firebuild may have been compiled with or without _TIME_BITS=64
, thus libfirebuild
now provides both variants on affected systems running glibc >= 34 to work safely with binaries using 64-bit and 32-bit time representation.
Many Linux distributions already stopped supporting 32-bit architectures, but Debian and Ubuntu still supports armhf, for example, where the Y2038 problem still applies. Both Debian and Ubuntu performed a transition rebuilding every library (and their reverse dependencies) with -D_FILE_OFFSET_BITS=64
set where the libraries exported symbols that changed when switching to 64-bit time representation (thanks to Steve Langasek for driving this!) . Thanks to the transition most programs are ready for 2038, but interposer libraries are trickier to fix and if you maintain one it might be a good idea to check if it works well both 32-bit and 64-bit libraries. Faketime, for example is not fixed yet, see #1064555.
Select passed through environment variables with regular expressions
Firebuild filters out most of the environment variables set when starting a build to make the build more reproducible and achieve higher cache hit rate. Extra environment variables to pass through can be specified on the command line one by one, but with many similarly named variables this may become hard to maintain. With regular expressions this just became easier:
firebuild -o 'env_vars.pass_through += "MY_VARS_.*"' my_build_command
If you are not interested in acceleration just would like to explore what the build does by generating a report you can simply pass all variables:
firebuild -r -o 'env_vars.pass_through += ".*"' my_build_command
Other highlights from the 0.8.3 release
- Fixed and nicer report in Chrome and other WebKit based browsers
- Support GLibc 2.39 by intercepting pidfd_spawn() and pidfd_spawnp()
- Even faster Rust build acceleration
For all the changes please check out the release page on GitHub!
(This post is also published on The Firebuild blog.)
December 05, 2024
E327 Nuvens À Moda Da Casa
Podcast Ubuntu Portugal
Fumando cachimbos em amena cavaqueira depois dos nossos afazeres diários, falámos descontraidamente sobre nuvens domésticas entre nuvens de fumo escaláveis; relembrámos os momentos bem agradáveis vividos no encontro da Comunidade em Sintra e no Centro Linux; trouxemos notícias sobre diapositivos feitos com MARP e Quarto; recordes de abas em Firefox; os últimos «ports» para Ubuntu Touch feitos por heróis nacionais e não falámos de mais coisas porque não havia tempo e o fumo já não deixava ver nada.
Já sabem: oiçam, subscrevam e partilhem!
- https://oracular.centrolinux.pt
- https://snapcraft.io/marp-cli-carroarmato0
- https://marp.app/
- https://quarto.org/docs/presentations/
- https://ubports.com/pt/blog/ubports-blogs-noticias-1/post/ubuntu-touch-ota-6-call-for-testing-3940
- https://9to5linux.com/ubuntu-touch-ota-6-rolls-out-with-wireless-display-improvements-bug-fixes
- https://x.com/UBports/status/1856264932930494877
- https://devices.ubuntu-touch.io/device/billie/
- https://x.com/UBports/status/1855312270466994196
- https://web.archive.org/web/20241201223909/https://ubports.com/pt/blog/ubports-blogs-noticias-1/post/ubuntu-touch-ota-7-focal-release-3943
- https://nlnet.nl/project/OpenAGPS/
- https://sfconservancy.org/news/2024/nov/29/openwrt-one-wireless-router-now-ships-black-friday/
- LoCo PT: https://loco.ubuntu.com/teams/ubuntu-pt/
- Nitrokey: https://shop.nitrokey.com/shop?aff_ref=3
- Mastodon: https://masto.pt/@pup
- Youtube: https://youtube.com/PodcastUbuntuPortugal
Apoios
Podem apoiar o podcast usando os links de afiliados do Humble Bundle, porque ao usarem esses links para fazer uma compra, uma parte do valor que pagam reverte a favor do Podcast Ubuntu Portugal. E podem obter tudo isso com 15 dólares ou diferentes partes dependendo de pagarem 1, ou 8. Achamos que isto vale bem mais do que 15 dólares, pelo que se puderem paguem mais um pouco mais visto que têm a opção de pagar o quanto quiserem. Se estiverem interessados em outros bundles não listados nas notas usem o link https://www.humblebundle.com/?partner=PUP e vão estar também a apoiar-nos.
Atribuição e licenças
Este episódio foi produzido por Diogo Constantino, Miguel e Tiago Carrondo e editado pelo Senhor Podcast. O website é produzido por Tiago Carrondo e o código aberto está licenciado nos termos da Licença MIT. (https://creativecommons.org/licenses/by/4.0/). A música do genérico é: “Won’t see it comin’ (Feat Aequality & N’sorte d’autruche)”, por Alpha Hydrae e está licenciada nos termos da CC0 1.0 Universal License. Este episódio e a imagem utilizada estão licenciados nos termos da licença: Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0), cujo texto integral pode ser lido aqui. Estamos abertos a licenciar para permitir outros tipos de utilização, contactem-nos para validação e autorização.
December 04, 2024
Hacked and tis the season for surgeries
Scarlett Gately Moore
I am still here. Sadly while I battle this insane infection from my broken arm I got back in July, the hackers got my blog. I am slowly building it back up. Further bad news is I have more surgeries, first one tomorrow. Furthering my current struggles I cannot start my job search due to hospitalization and recovery. Please consider a donation. https://gofund.me/6e99345d
On the open source work front, I am still working on stuff, mostly snaps ( Apps 24.08.3 released )
Thank you everyone that voted me into the Ubuntu Community Council!
I am trying to stay positive, but it seems I can’t catch a break. I will have my computer in the hospital and will work on what I can. Have a blessed day and see you soon.
Scarlett
December 03, 2024
Introducing Launchpad Bug Templates
Launchpad News
The new feature bug templates in Launchpad aims to streamline the bug reporting process, making it more efficient for both users and project maintainers.
In the past, Launchpad provided only a basic description field for filling bug reports. This often led to incomplete or vague submissions, as users may not include essential details or steps to reproduce an issue. This could slow down the debugging process when fixing bugs.
To improve this, we are introducing bug templates. These allow project maintainers to guide users when reporting bugs. By offering a structured template, users are prompted to provide all the necessary information, which helps to speed up the development process.
To start using bug templates in your project, simply follow these steps:
- Access your project’s bug page view.
- Select ‘Configure bugs’.
- A field showing the bug template will prompt you to fill in your desired template.
- Save the changes. The template will now be available to users when they report a new bug for your project.
For now, only a default bug template can be set per project. Looking ahead, the idea is to expand this by introducing multiple bug templates per project, as well as templates for other content types such as merge proposals or answers. This will allow project maintainers to define various templates for different purposes, making the open-source collaboration process even more efficient.
Additionally, we will introduce Markdown support, allowing maintainers to create structured and visually clear templates using features such as headings, lists, or code blocks.
December 01, 2024
Free software activity in November 2024
Colin Watson
Most of my Debian contributions this month were sponsored by Freexian.
You can also support my work directly via Liberapay.
Conferences
I attended MiniDebConf Toulouse 2024, and the MiniDebCamp before it. Most of my time was spent with the Freexian folks working on debusine; Stefano gave a talk about its current status with a live demo (frantically fixed up over the previous couple of days, as is traditional) and with me and others helping to answer questions at the end. I also caught up with some people I haven’t seen in ages, ate a variety of delicious cheeses, and generally had a good time. Many thanks to the organizers and sponsors!
After the conference, Freexian collaborators spent a day and a half doing some planning for next year, and then went for an afternoon visiting the Cité de l’espace.
Rust team
I upgraded these packages to new upstream versions, as part of upgrading pydantic and rpds-py:
- rust-archery
- rust-jiter (noticing an upstream test bug in the process)
- rust-pyo3 (fixing CVE-2024-9979)
- rust-pyo3-build-config
- rust-pyo3-ffi
- rust-pyo3-macros
- rust-pyo3-macros-backend
- rust-regex
- rust-regex-automata
- rust-regex
- rust-serde
- rust-serde-derive
- rust-serde-json
- rust-speedate
- rust-triomphe
Python team
Last month, I mentioned that we still need to work out what to do about the multipart vs. python-multipart name conflict in Debian (#1085728). We eventually managed to come up with an agreed plan; Sandro has uploaded a renamed binary package to experimental, and I’ve begun work on converting reverse-dependencies (asgi-csrf, fastapi, python-curies, and starlette done so far). There’s a bit more still to do, but I expect we can finish it soon.
I fixed problems related to adding Python 3.13 support in:
- coreapi
- git-repo-updater
- offlineimap3
- ptyprocess
- pytest-testinfra
- python-formencode
- python-iniparse
- python-line-profiler
- python-parameterized
- python-sdjson (contributed upstream)
- python-testfixtures (contributed upstream)
- python-venusian
- sphinx-a4doc (contributed upstream, and I also made a small antlr4 improvement)
- webpy
I fixed some packaging problems that resulted in failures any time we add a new Python version to Debian:
I fixed other build/autopkgtest failures in:
- psycopg3 (contributed upstream)
- python-aiohttp
- python-cotengrust
- python-distutils-extra
- python-formencode
- python-openapi-schema-validator
- python-openapi-spec-validator
- sphinxcontrib-towncrier
I packaged python-quart-trio, needed for a new upstream version of python-urllib3, and contributed a small packaging tweak upstream.
I backported a twisted fix that caused problems in other packages, including breaking debusine‘s tests.
I disentangled some upstream version confusion in python-catalogue, and upgraded to the current upstream version.
I upgraded these packages to new upstream versions:
- aioftp (fixing a Python 3.13 failure)
- ansible-core
- ansible
- debugpy
- jsonpickle
- manuel
- psycopg2
- pydantic-core
- pydantic
- pydantic-settings
- pymssql (fixing a Python 3.13 failure)
- pyodbc (fixing a Python 3.13 failure)
- python-argh (fixing a Python 3.13 failure)
- python-boltons (fixing a Python 3.13 failure)
- python-channels-redis
- python-colorlog (fixing a Python 3.13 failure)
- python-django-pgtrigger
- python-line-profiler
- python-pathvalidate (fixing a Python 3.13 failure)
- python-plac (fixing a Python 3.13 failure)
- python-precis-i18n
- python-pure-eval (fixing a Python 3.13 failure)
- python-pythonjsonlogger (contributing a small packaging fix upstream, as well as a test fix to jupyter-events)
- python-rdata (fixing a Python 3.13 failure)
- python-semantic-release
- python-telethon (fixing a Python 3.13 failure, and contributing some test fixes upstream)
- python-tornado (fixing CVE-2024-52804)
- python-trio (fixing a Python 3.13 failure)
- python-trustme
- python-typeguard
- python-urllib3 (fixing CVE-2024-37891 and a Python 3.13 failure, and requiring some shenanigans with its hypercorn test-dependency)
- python-zipp
- quart (fixing CVE-2024-49767)
- rpds-py (fixing a build failure)
- sen
- sqlparse
- stravalib
- transaction
- waitress
- zope.interface
Other small fixes
I contributed Incus support to needrestart upstream.
In response to Helmut’s Cross building talk at MiniDebConf Toulouse, I fixed libfilter-perl to support cross-building (5b4c2e10, f9788c27).
I applied a patch to move aliased files from /
to /usr
in iprutils
(#1087733).
I adjusted debconf to use the new /usr/lib/apt/apt-extracttemplates
path
(#1087523).
I upgraded putty to 0.82.
November 18, 2024
Announcing Incus 6.7
Stéphane Graber
The Incus team is pleased to announce the release of Incus 6.7!
This is another one of those pretty well rounded releases with new features and improvements for everyone from standalone users to those running a small homelab all the way to large scale cluster users, there’s something for everyone!
The highlights for this release are:
- Easy access to the Incus web interface
- Automatic cluster re-balancing
- DHCP renewal for OCI containers
- Partial instance/volume refresh
- Configurable columns, formatting and refresh time for incus top
- Support for DHCP address ranges in OVN networks
- Changing of parent device for physical networks
- Aditional QMP helpers in QEMU scriptlet
- Additional QEMU log file for QMP commands
- New get_instances_count command for placement scriptlet
- Support of formatting in incus admin sql command
The full announcement and changelog can be found here.
And for those who prefer videos, here’s the release overview video:
You can take the latest release of Incus up for a spin through our online demo service at: https://linuxcontainers.org/incus/try-it/
And as always, my company is offering commercial support on Incus, ranging from by-the-hour support contracts to one-off services on things like initial migration from LXD, review of your deployment to squeeze the most out of Incus or even feature sponsorship. You’ll find all details of that here: https://zabbly.com/incus
Donations towards my work on this and other open source projects is also always appreciated, you can find me on Github Sponsors, Patreon and Ko-fi.
Enjoy!
November 17, 2024
November 12, 2024
Complex for Whom?
Paul Tagliamonte
In basically every engineering organization I’ve ever regarded as particularly high functioning, I’ve sat through one specific recurring conversation which is not – a conversation about “complexity”. Things are good or bad because they are or aren’t complex, architectures needs to be redone because it’s too complex – some refactor of whatever it is won’t work because it’s too complex. You may have even been a part of some of these conversations – or even been the one advocating for simple light-weight solutions. I’ve done it. Many times.
Rarely, if ever, do we talk about complexity within its rightful context – complexity for whom. Is a solution complex because it’s complex for the end user? Is it complex if it’s complex for an API consumer? Is it complex if it’s complex for the person maintaining the API service? Is it complex if it’s complex for someone outside the team maintaining it to understand? Complexity within a problem domain I’ve come to believe, is fairly zero-sum – there’s a fixed amount of complexity in the problem to be solved, and you can choose to either solve it, or leave it for those downstream of you to solve that problem on their own.
That being said, while I believe there is a lower bound in complexity to contend with for a problem, I do not believe there is an upper bound to the complexity of solutions possible. It is always possible, and in fact, very likely that teams create problems for themselves while trying to solve a problem. The rest of this post is talking to the lower bound. When getting feedback on an early draft of this blog post, I’ve been informed that Fred Brooks coined a term for what I call “lower bound complexity” – “Essential Complexity”, in the paper “No Silver Bullet—Essence and Accident in Software Engineering”, which is a better term and can be used interchangeably.
Complexity Culture
In a large enough organization, where the team is high functioning enough to have and maintain trust amongst peers, members of the team will specialize. People will begin to engage with subsets of the work to be done, and begin to have their efficacy measured against that part of the organization’s problems. Incentives shift, and over time it becomes increasingly likely that two engineers may have two very different priorities when working on the same system together. Someone accountable for uptime and tasked with responding to outages will begin to resist changes. Someone accountable for rapidly delivering features will resist gates between them and their users. Companies (either wittingly or unwittingly) will deal with this by tasking engineers with both production (feature development) and operational tasks (maintenance), so the difference in incentives isn’t usually as bad as it could be.
When we get a bunch of folks from far-flung corners of an organization in a room, fire up a slide deck and throw up some aspirational to-be architecture diagram in order to get a sign-off to solve some problem (be it someone needs a credible promotion packet, new feature needs to get delivered, or the system has begun to fail and needs fixing), the initial reaction will, more often than I’d like, start to devolve into a discussion of how this is going to introduce a bunch of complexity, going to be hard to maintain, why can’t you make it less complex?
Right around here is when I start to try and contextualize the conversation happening around me – understand what complexity is that being discussed, and understand who is taking on that burden. Think about who should be owning that problem, and work through the tradeoffs involved. Is it best solved here, or left to consumers (be them other systems, developers, or users). Should something become an API call’s optional param, taking on all the edge-cases and on, or should users have to implement the logic using the data you return (leaving everyone else to take on all the edge-cases and maintenance)? Should you process the data, or require the user to preprocess it for you?
Frequently it’s right to make an active and explicit decision to simplify and leave problems to be solved downstream, since they may not actually need to be solved – or perhaps you expect consumers will want to own the specifics of how the problem is solved, in which case you leave lots of documentation and examples. Many other times, especially when it’s something downstream consumers are likely to hit, it’s best solved internal to the system, since the only thing that can come of leaving it unsolved are bugs, frustration and half-correct solutions. This is a grey-space of tradeoffs, not a clear decision tree. No one wants the software manifestation of a katamari ball or a junk drawer, nor does anyone want a half-baked service unable to handle the simplest use-case.
Head-in-sand as a Service
Popoffs about how complex something is, are, to a first approximation, best
understood as meaning “complicated for the person making comments”. A lot of
the #thoughtleadership
believe that an AWS hosted EKS k8s
cluster running
images built by CI talking to an AWS hosted PostgreSQL RDS is not complex.
They’re right. Mostly right. This is less complex – less complex for them.
It’s not, however, without complexity and its own tradeoffs – it’s just
complexity that they do not have to deal with. Now they don’t have to
maintain machines that have pesky operating systems or hard drive failures.
They don’t have to deal with updating the version of k8s
, nor ensuring the
backups work. No one has to push some artifact to prod manually. Deployments
happen unattended. You click a button and get a cluster.
On the other hand, developers outside the ops function need to deal with
troubleshooting CI, debugging access control rules encoded in turing complete
YAML, permissions issues inside the cluster due to whatever the fuck a service
mesh is, everyone needs to learn how to use some k8s
tools they only actually
use during a bad day, likely while doing some x.509
troubleshooting to
connect to the cluster (an internal only endpoint; just port forward it) – not
to mention all sorts of rules to route packets to their project (a single
repo’s binary being run in 3 containers on a single vm host).
Beyond that, there’s the invisible complexity – complexity on the interior of
a service you depend on. I think about the dozens of teams maintaining the EKS
service (which is either run on EC2 instances, or alternately, EC2 instances in
a trench coat, moustache and even more shell scripts), the RDS service (also
EC2 and shell scripts, but this time accounting for redundancy, backups,
availability zones), scores of hypervisors pulled off the shelf (xen
, kvm
)
smashed together with the ones built in-house (firecracker
, nitro
, etc)
running on hardware that has to be refreshed and maintained continuously. Every
request processed by network ACL rules, AWS IAM rules, security group rules,
using IP space announced to the internet wired through IXPs directly into ISPs.
I don’t even want to begin to think about the complexity inherent in how those
switches are designed. Shitloads of complexity to solve problems you may or
may not have, or even know you had.
What’s more complex? An app running in an in-house 4u server racked in the office’s telco closet in the back running off the office Verizon line, or an app running four hypervisors deep in an AWS datacenter? Which is more complex to you? What about to your organization? In total? Which is more prone to failure? Which is more secure? Is the complexity good or bad? What type of Complexity can you manage effectively? Which threaten the system? Which threaten your users?
COMPLEXIVIBES
This extends beyond Engineering. Decisions regarding “what tools are we able to use” – be them existing contracts with cloud providers, CIO mandated SaaS products, a list of the only permissible open source projects – will incur costs in terms of expressed “complexity”. Pinning open source projects to a fixed set makes SBOM production “less complex”. Using only one SaaS provider’s product suite (even if its terrible, because it has all the types of tools you need) makes accreditation “less complex”. If all you have is a contract with Pauly T’s lowest price technically acceptable artisinal cloudary and haberdashery, the way you pay for your compute is “less complex” for the CIO shop, though you will find yourself building your own hosted database template, mechanism to spin up a k8s cluster, and all the operational and technical burden that comes with it. Or you won’t and make it everyone else’s problem in the organization. Nothing you can do will solve for the fact that you must now deal with this problem somewhere because it was less complicated for the business to put the workloads on the existing contract with a cut-rate vendor.
Suddenly, the decision to “reduce complexity” because of an existing contract vehicle has resulted in a huge amount of technical risk and maintenance burden being onboarded. Complexity you would otherwise externalize has now been taken on internally. With large enough organizations (specifically, in this case, I’m talking about you, bureaucracies), this is largely ignored or accepted as normal since the personnel cost is understood to be free to everyone involved. Doing it this way is more expensive, more work, less reliable and less maintainable, and yet, somehow, is, in a lot of ways, “less complex” to the organization. It’s particularly bad with bureaucracies, since screwing up a contract will get you into much more trouble than delivering a broken product, leaving basically no reason for anyone to care to fix this.
I can’t shake the feeling that for every story of technical mandates gone awry, somewhere just out of sight there’s a decisionmaker optimizing for what they believe to be the least amount of complexity – least hassle, fewest unique cases, most consistency – as they can. They freely offload complexity from their accreditation and risk acceptance functions through mandates. They will never have to deal with it. That does not change the fact that someone does.
TC;DR (TOO COMPLEX; DIDN’T REVIEW)
We wish to rid ourselves of systemic Complexity – after all, complexity is bad, simplicity is good. Removing upper-bound own-goal complexity (“accidental complexity” in Brooks’s terms) is important, but once you hit the lower bound complexity, the tradeoffs become zero-sum. Removing complexity from one part of the system means that somewhere else - maybe outside your organization or in a non-engineering function - must grow it back. Sometimes, the opposite is the case, such as when a previously manual business processes is automated. Maybe that’s a good idea. Maybe it’s not. All I know is that what doesn’t help the situation is conflating complexity with everything we don’t like – legacy code, maintenance burden or toil, cost, delivery velocity.
- Complexity is not the same as proclivity to failure. The most reliable systems I’ve interacted with are unimaginably complex, with layers of internal protection to prevent complete failure. This has its own set of costs which other people have written about extensively.
- Complexity is not cost. Sometimes the cost of taking all the complexity in-house is less, for whatever value of cost you choose to use.
- Complexity is not absolute. Something simple from one perspective may be wildly complex from another. The impulse to burn down complex sections of code is helpful to have generally, but sometimes things are complicated for a reason, even if that reason exists outside your codebase or organization.
- Complexity is not something you can remove without introducing complexity elsewhere. Just as not making a decision is a decision itself; choosing to require someone else to deal with a problem rather than dealing with it internally is a choice that needs to be considered in its full context.
Next time you’re sitting through a discussion and someone starts to talk about all the complexity about to be introduced, I want to pop up in the back of your head, politely asking what does complex mean in this context? Is it lower bound complexity? Is this complexity desirable? Is what they’re saying mean something along the lines of I don’t understand the problems being solved, or does it mean something along the lines of this problem should be solved elsewhere? Do they believe this will result in more work for them in a way that you don’t see? Should this not solved at all by changing the bounds of what we should accept or redefine the understood limits of this system? Is the perceived complexity a result of a decision elsewhere? Who’s taking this complexity on, or more to the point, is failing to address complexity required by the problem leaving it to others? Does it impact others? How specifically? What are you not seeing?
What can change?
What should change?
November 04, 2024
My Keys
Stuart Langridge
I have a problematic relationship with keys.
Well, that's not true. I have a problematic relationship with key rings. For some reason, my pockets are a violently hostile environment for things I put in them. I don't really understand why this is, but it's true. Keyrings bend out of shape; the concentric rings separate, and my actual keys fall off of them. People have expressed scepticism about this in the past, and they've been wrong and I've been right. The last time I complained about this, I thought I'd come up with a solution where I bought a keyring which was a tiny padlock. It lasted three days before a bolt sheared off. You can see the whole thing on posts made to twitter.
At that point most people would give up, or be sad, or just live with split rings continually letting them down. But most people don't have a dad who is a king of engineering.
My dad made me this.
It's a keyring. It's a solid block of brass with the middle cut out, so it looks like a very shallow "U", or like three sides of a long rectangle. There's a hole drilled in each of the short ends, and a long bolt is threaded through those holes. On one end of the bolt is a nut, tight against the outside of the "U", and the bolt protrudes out about an inch where there's another, locking, nut. All the keys are hung from the bolt. To add a new key, I undo the locking nut on the end, undo the tight nut, pull the bolt out, hang another key from it, and then do everything back up. It's brilliant. I've not had a single problem with it.
Those of you carefully studying the picture will notice that there is writing on the brass "U". (And will also notice that I've blacked out the details of the actual keys, because you can cut a key based on a picture, and I'm not stupid.) That's engraving, which mentions my website, so if I lose my keys (which I am really careful to not do1) then whoever finds them can get in touch with me to tell me that happened but my address is not on the keys, so a nefarious finder gets less benefit from it.
I love my keyring. It's the best. I do not know why more keyrings are not like this. It works just like a normal keyring (I have non-key things on mine, such as a USB stick and a tiny flashlight, but they would go on a regular split-ring keyring as well), but it doesn't just fail all the time like normal ones do. I surely can't be the only person who experiences this? Anyway, I don't mind, 'cos I have the solution. Cheers, dad. Maybe I should make this a product or something.
- the historical version of checking your pockets, as a man, was to feel for "spectacles, testicles, wallet, and watch" -- this was actually a ribald mnemonic for how to cross yourself as a Catholic, but this modern day man checks his pockets for keys, wallet, and phone in the same way to check they're not lost ↩
November 01, 2024
Free software activity in October 2024
Colin Watson
Almost all of my Debian contributions this month were sponsored by Freexian.
You can also support my work directly via Liberapay.
Ansible
I noticed that Ansible had fallen out of Debian testing due to autopkgtest failures. This seemed like a problem worth fixing: in common with many other people, we use Ansible for configuration management at Freexian, and it probably wouldn’t make our sysadmins too happy if they upgraded to trixie after its release and found that Ansible was gone.
The problems here were really just slogging through test failures in both
the ansible-core
and ansible
packages, but their test suites are large
and take a while to run so this took some time. I was able to contribute a
few small fixes to various upstreams in the process:
- test: Make git archive prefix fit in 32-bit
ssize_t
(though upstream went for a different approach) - test: replace more deprecated
assertEquals
- Fix import error on Python 3.13
- Make
test_start_daemon_with_no_mock
less flaky
This should now get back into testing tomorrow.
OpenSSH
Martin-Éric Racine
reported that ssh-audit
didn’t list the ext-info-s
feature as being available in Debian’s OpenSSH
9.2 packaging in bookworm, contrary to what OpenSSH upstream said on their
specifications page at the time. I
spent some time looking into this and realized that upstream was mistakenly
saying that implementations of ext-info-c
and ext-info-s
were added at
the same time, while in fact ext-info-s
was added rather later.
ssh-audit
now has clearer output, and the OpenSSH maintainers have
corrected their specifications page.
I looked into a report of an ssh
failure in certain cases when using GSS-API key exchange (which is a Debian
patch). Once again, having integration
tests was a huge win here: the affected
scenario is quite a fiddly one, but I was able to set it up in the
test,
and thereby make sure it doesn’t regress in future. It still took me a
couple of hours to get all the details right, but in the past this sort of
thing took me much longer with a much lower degree of confidence that the
fix was correct.
On upstream’s advice, I cherry-picked some key exchange fixes needed for big-endian architectures.
Python team
I packaged python-evalidate, needed for a new upstream version of buildbot.
The Python 3.13 transition rolls on. I fixed problems related to it in htmlmin, humanfriendly, postgresfixture (contributed upstream), pylint, python-asyncssh (contributed upstream), python-oauthlib, python3-simpletal, quodlibet, zope.exceptions, and zope.interface.
A trickier Python 3.13 issue involved the cgi
module. Years ago I ported
zope.publisher to the multipart module because
cgi.FieldStorage
was broken in some situations, and as a result I got a
recommendation into Python’s
“dead batteries” PEP 594.
Unfortunately there turns out to be a name conflict between multipart and
python-multipart on
PyPI;
python-multipart upstream has been working to disentangle
this, though we still
need to work out what to do in Debian.
All the same, I needed to fix
python-wadllib and multipart seemed like
the best fit; I contributed a port
upstream
and temporarily copied multipart into Debian’s python-wadllib source package
to allow its tests to pass. I’ll come back and fix this properly once we
sort out the multipart vs. python-multipart packaging.
tzdata moved some timezone definitions to tzdata-legacy, which has broken a number of packages. I added tzdata-legacy build-dependencies to alembic and python-icalendar to deal with this in those packages, though there are still some other instances of this left.
I tracked down an nltk regression that caused build failures in many other packages.
I fixed Rust crate versioning issues in pydantic-core, python-bcrypt, and python-maturin (mostly fixed by Peter Michael Green and Jelmer Vernooij, but it needed a little extra work).
I fixed other build failures in entrypoints, mayavi2, python-pyvmomi (mostly fixed by Alexandre Detiste, but it needed a little extra work), and python-testing.postgresql (ditto).
I fixed python3-simpletal to tolerate future versions of dh-python that will drop their dependency on python3-setuptools.
I fixed broken symlinks in python-treq.
I removed (build-)depends on python3-pkg-resources from alembic, autopep8, buildbot, celery, flufl.enum, flufl.lock, python-public, python-wadllib (contributed upstream), pyvisa, routes, vulture, and zodbpickle (contributed upstream).
I upgraded astroid, asyncpg (fixing a Python 3.13 failure and a build failure), buildbot (noticing an upstream test bug in the process), dnsdiag, frozenlist, netmiko (fixing a Python 3.13 failure), psycopg3, pydantic-settings, pylint, python-asyncssh, python-bleach, python-btrees, python-cytoolz, python-django-pgtrigger, python-django-test-migrations, python-gssapi, python-icalendar, python-json-log-formatter, python-pgbouncer, python-pkginfo, python-plumbum, python-stdlib-list, python-tokenize-rt, python-treq (fixing a Python 3.13 failure), python-typeguard, python-webargs (fixing a build failure), pyupgrade, pyvisa, pyvisa-py (fixing a Python 3.13 failure), toolz, twisted, vulture, waitress (fixing CVE-2024-49768 and CVE-2024-49769), wtf-peewee, wtforms, zodbpickle, zope.exceptions, zope.interface, zope.proxy, zope.security, and zope.testrunner to new upstream versions.
I tried to fix a regression in python-scruffy, but I need testing feedback.
I requested removal of python-testing.mysqld.
October 22, 2024
Two Plumbers
Stuart Langridge
In a land far away, there were two brothers, two plumbers. To preserve their anonymity, we'll call them... Mario and Luigi. Their mother, a kind and friendly woman, and their father, a man with (by the laws of averages and genetics) a truly gargantuan moustache, raised them both to be kind and friendly (and moustachioed) in their turn. There was enough work in the town to keep both the plumbers busy, and they each grew through apprentice to journeyman to experience and everyone liked them. They both cared about the job, about their clients, and they each did good work, always going the extra mile, doing more than was necessarily asked for, putting in an extra hour to tighten that pipe or fit a better S-bend or clean up the poor workmanship of lesser craftsmen and cowboys. They were happy. Even their rivalry for each job was good-humoured, a friendly source of amusement to them and to the town. Sometimes people would flip a coin to choose which to ring, having no way to choose between them, and Mario would laugh and suggest that he should have two-headed coins made, or Luigi would laugh and say that that ought to make it his turn next.
But there came a time of downturn, when the people of the town had to hold tighter to their purses, and fewer called out for plumbers. And Luigi, after much thought, decided to take a job with Bowser's, the big plumbing conglomerate from the city. He was worried: the big company were often slapdash or inexperienced in their work, and discourteous or evasive to their clients, and more interested in bottom lines than hot water lines. But they paid extremely well, and they had the latest tools, and there was security in having a contract and a title and a boss. Besides, Bowser's worked for so many more people that Luigi's own skills could only help that many more. Maybe he could even teach them something about quality, and craftsmanship, and care. He suggested to Mario that they both joined, and Mario thought hard about it, and eventually decided not to, though it was a close-run thing. Both the brothers shook hands on it, respecting one another's decision, although in the silence of their hearts each was a little disappointed in the other.
Luigi did well at Bowser's. He was right about the latest tools, and about the pay, and about the security. And he was partially right about teaching the big company something about quality. His work was often better than his colleagues, sometimes through expertise but most often because he tried harder: he loved the work, and wanted to do well, and was kind and friendly when he could be. But sometimes, try though he might, the time wasn't there, or the parts weren't in the van, and these things were not his fault; someone else at the big company had cut corners on their job and that forced Luigi to cut corners on his and make people sad and angry, or put in more time to fix it than he would have spent doing it all correctly himself in the first place. He pushed hard inside the company to fix these things, and he had some successes; a policy was written suggesting that employees work harder to improve customer happiness, and many customers across the land were made a little happier as a result. Luigi won an award. He trained some apprentices, and many of his little ways of making people happier and the job better were adopted into the company training scheme. One time he went home after another argument with his boss about the things that were not adopted, and that night he looked enviously out of their window at his brother's house across the street, thinking that it would be a fine thing to not have a boss who stopped you from doing things right.
Mario did well working for himself. The time of downturn ended and things began to pick up again, maybe not quite to where they had been but nearly there for all that, and the phone calls and messages came in once more. Everyone was pleased to see him, and although he maybe took a little longer than the men from the big company, his work was never slapdash, always taking the time to do it right. And he had less money, but he really didn't mind, or begrudge it; he had enough to get by, and he loved the work, and wanted to do well, and was kind and friendly. He did envy his brother's toolbox, though, all the latest gear while Mario himself made do with things a little older, a little rustier, but they were all good quality tools that he understood, and the work was as good and better. In November one year a very expensive plumber's inspection camera was stolen from his brother's van, and Mario thought that it would have been great to have such a thing and maybe he would have taken better care, and then he felt guilty about thinking that of his brother. He felt guiltier still when on Christmas morning he opened the box from Luigi to find an expensive inspection camera in it. But then his brother winked at him and put a finger to his lips, and all was well between them again. One time Mario was up to his waist in the drain outside a house, raindrops rattling on his hat and cursing the god who invented backflow, when he saw his brother drive past all unknowing in his modern van, windows wound up and singing along with the radio, and he looked enviously after the van's lights in the storm, thinking that it would be a fine thing to have just a notch more comfort and influence and two fewer wet knees.
October 20, 2024
2024 LiveCD Memory Usage Compare
Bryan Quigley
I am using pretty much the exact same setup I did in 2020. Let's see who is more efficient in a live session!
But first let's take a look at the image sizes:
Charge Open Movie is what I viewed if I can make it to YouTube.
I decided to be more selective and remove those that did very porly at 1.5G, which was most.
- Ubuntu - booted but desktop not stable, took 1.5 minutes to load Firefox
- Xubuntu-minimal - does not include a web browser so can't further test. Snap is preinstaled even though no apps are - but trying to install a web browser worked but couldn't start.
- Manjaro KDE - Desktop loads, but browser doesn't
- Xubuntu - laggy when Firefox is opened, can't load sites
- Ubuntu Mate -laggy when Firefox is opened, can't load sites
- Kubuntu - laggy when Firefox is opened, can't load sites
- Linux Mint 22 - desktop loads, browsers isn't responsive
Fedora video is a bit laggy, but watchable.. EndlessOS with Chromium is the most smooth and resonsive watching YouTube.
For fun let's look at startup time with 2GB (with me hitting buttons as needed to open a folder)
Conclusion
- Lubuntu lowered it's memory usage from 2020 for loading a desktop 585M to 450M! Kudos to Lubuntu team!
- Both Fedora and Endless desktops worked in lower memory then 2020 too!
- Lubuntu, Fedora and Endless all used Zram.
- Chromium has definitely improved it's memory usage as last time Endless got dinged for using it. Now it appears to work better then Firefox.
Notes:
- qemu-system-x86_64 -enable-kvm -cdrom lubuntu-24.04.1-desktop-amd64.iso -m 1.5G -smp 4 -cpu host -vga virtio --full-screen
- Screen size was set to 1080p/60Hz.
- I tried to reproduce 585M on Lubuntu 20.04 build, but it failed on anything below 1G.
- Getting out of full screen on YouTube apparently is an intensive task. Dropped testing that.
- All Ubuntu was 24.04.1 LTS.
October 15, 2024
Waiting for a Linux system to be online
Lukas Märdian
What is an “online” system?
Networking is a complex topic, and there is lots of confusion around the definition of an “online” system. Sometimes the boot process gets delayed up to two minutes, because the system still waits for one or more network interfaces to be ready. Systemd provides the network-online.target that other service units can rely on, if they are deemed to require network connectivity. But what does “online” actually mean in this context, is a link-local IP address enough, do we need a routable gateway and how about DNS name resolution?
The requirements for an “online” network interface depend very much on the services using an interface. For some services it might be good enough to reach their local network segment (e.g. to announce Zeroconf services), while others need to reach domain names (e.g. to mount a NFS share) or reach the global internet to run a web server. On the other hand, the implementation of network-online.target varies, depending on which networking daemon is in use, e.g. systemd-networkd-wait-online.service or NetworkManager-wait-online.service. For Ubuntu, we created a specification that describes what we as a distro expect an “online” system to be. Having a definition in place, we are able to tackle the network-online-ordering issues that got reported over the years and can work out solutions to avoid delayed boot times on Ubuntu systems.
In essence, we want systems to reach the following networking state to be considered online:
- Do not wait for “optional” interfaces to receive network configuration
- Have IPv6 and/or IPv4 “link-local” addresses on every network interface
- Have at least one interface with a globally routable connection
- Have functional domain name resolution on any routable interface
A common implementation
NetworkManager and systemd-networkd are two very common networking daemons used on modern Linux systems. But they originate from different contexts and therefore show different behaviours in certain scenarios, such as wait-online. Luckily, on Ubuntu we already have Netplan as a unification layer on top of those networking daemons, that allows for common network configuration, and can also be used to tweak the wait-online logic.
With the recent release of Netplan v1.1 we introduced initial functionality to tweak the behaviour of the systemd-networkd-wait-online.service, as used on Ubuntu Server systems. When Netplan is used to drive the systemd-networkd backend, it will emit an override configuration file in /run/systemd/system/systemd-networkd-wait-online.service.d/10-netplan.conf, listing the specific non-optional interfaces that should receive link-local IP configuration. In parallel to that, it defines a list of network interfaces that Netplan detected to be potential global connections, and waits for any of those interfaces to reach a globally routable state.
Such override config file might look like this:
[Unit]
ConditionPathIsSymbolicLink=/run/systemd/generator/network-online.target.wants/systemd-networkd-wait-online.service
[Service]
ExecStart=
ExecStart=/lib/systemd/systemd-networkd-wait-online -i eth99.43:carrier -i lo:carrier -i eth99.42:carrier -i eth99.44:degraded -i bond0:degraded
ExecStart=/lib/systemd/systemd-networkd-wait-online --any -o routable -i eth99.43 -i eth99.45 -i bond0
In addition to the new features implemented in Netplan, we reached out to upstream systemd, proposing an enhancement to the systemd-networkd-wait-online service, integrating it with systemd-resolved to check for the availability of DNS name resolution. Once this is implemented upstream, we’re able to fully control the systemd-networkd backend on Ubuntu Server systems, to behave consistently and according to the definition of an “online” system that was lined out above.
Future work
The story doesn’t end there, because Ubuntu Desktop systems are using NetworkManager as their networking backend. This daemon provides its very own nm-online utility, utilized by the NetworkManager-wait-online systemd service. It implements a much higher-level approach, looking at the networking daemon in general instead of the individual network interfaces. By default, it considers a system to be online once every “autoconnect” profile got activated (or failed to activate), meaning that either a IPv4 or IPv6 address got assigned.
There are considerable enhancements to be implemented to this tool, for it to be controllable in a fine-granular way similar to systemd-networkd-wait-online, so that it can be instructed to wait for specific networking states on selected interfaces.
A note of caution
Making a service depend on network-online.target is considered an antipattern in most cases. This is because networking on Linux systems is very dynamic and the systemd target can only ever reflect the networking state at a single point in time. It cannot guarantee this state to be remained over the uptime of your system and has the potentially to delay the boot process considerably. Cables can be unplugged, wireless connectivity can drop, or remote routers can go down at any time, affecting the connectivity state of your local system. Therefore, “instead of wondering what to do about network.target, please just fix your program to be friendly to dynamically changing network configuration.” [source].
October 14, 2024
Kubuntu 24.10 Released, KDE Snaps at 24.08.2, and I lived to tell you about it!
Scarlett Gately Moore
Sorry my blog updates have been MIA. Let me tell you a story…
As some of you know, 3 months ago I was in a no fault car accident. Thankfully, the only injury was I ended up with a broken arm. ER sends me home in a sling and tells me it was a clean break and it will mend itself in no time. After a week of excruciating pain I went to my follow up doctor appointment, and with my x-rays in hand, the doc tells me it was far from a clean break and needs surgery. So after a week of my shattered bone scraping my nerves and causing pain I have never felt before, I finally go in for surgery! They put in a metal plate with screws to hold the bone in place so it can properly heal. The nerve pain was gone, so I thought I was on the mend. Some time goes by and the swelling still has not subsided, the doctors are not as concerned about this as I am, so I carry on until it becomes really inflamed and developed fever blisters. After no success in reaching the doctors office my husband borrows the neighbors car and rushes me to the ER. Good thing too, I had an infection. So after a 5 day stay in the hospital, they sent us home loaded with antibiotics and trained my husband in wound packing. We did everything right, kept the place immaculate, followed orders with the wound care, took my antibiotics, yet when they ran out there was still no sign of relief, or healing. Went to doctors and they gave me another month supply of antibiotics. Two days after my final dose my arm becomes inflamed again and with extra spectacular levels of pain to go with it. I call the doctor office… They said to come in on my appointment day ( 4 days away ). I asked, “You aren’t concerned with this inflammation?”, to which they replied, “No.”. Ok, maybe I am over reacting and it’s all in my head, I can power through 4 more days. The following morning my husband observed fever blisters and the wound site was clearly not right, so once again off we go to the ER. Well… thankfully we did. I was in Sepsis and could have died… After deliberating with the doctor on the course of action for treatment, the doctor accepted our plea to remove the plate, rather than tighten screws and have me drive 100 miles to hospital everyday for iv antibiotics (Umm I don’t have a car!?) So after another 4 day stay I am released into the world, alive and well. I am happy to report, the swelling is almost gone, the pain is minimal, and I am finally healing nicely. I am still in a sling and I have to be super careful and my arm was not fully knitted. So with that I am bummed to say, no traveling for me, no Ubuntu Summit
I still need help with that car, if it weren’t for our neighbor, this story would have ended much differently.
Despite my tragic few months for my right arm, my left arm has been quite busy. Thankfully I am a lefty! On to my work progress report.
Kubuntu:
KDE Snaps:
All release service snaps are done! Save a few problematic ones still WIP.. I have released 24.08.2 which you can find here:
https://snapcraft.io/publisher/kde
I completed the qt6 and KDE frameworks 6 content packs for core24
Snapcraft:
I have a PR in for kde-neon-6 extension core24 support.
That’s all for now. Thanks for stopping by!
Announcing Incus 6.6
Stéphane Graber
A slightly less busy release this time, mostly due to traveling to the Linux Plumbers Conference and associated events a few weeks ago.
But still far from a boring release. On top of the usual bugfix and performance improvements, we’re getting a number of nice additions for virtual machines, improved clustered LVM support, improvements to incus-migrate and a number of new network features!
The highlights for this release are:
- OS info for virtual machines
- Console history for virtual machines
- Ability to create clustered LVM volume groups
- QCOW2 and VMDK support in incus-migrate
- Configurable macvlan mode
- Load-balancer health information
- External interfaces in OVN networks
- Parallel cluster evacuation/restoration
The full announcement and changelog can be found here.
And for those who prefer videos, here’s the release overview video:
You can take the latest release of Incus up for a spin through our online demo service at: https://linuxcontainers.org/incus/try-it/
And as always, my company is offering commercial support on Incus, ranging from by-the-hour support contracts to one-off services on things like initial migration from LXD, review of your deployment to squeeze the most out of Incus or even feature sponsorship. You’ll find all details of that here: https://zabbly.com/incus
Donations towards my work on this and other open source projects is also always appreciated, you can find me on Github Sponsors, Patreon and Ko-fi.
Enjoy!
October 13, 2024
Redis: The Powerhouse Behind Modern Databases
Faizul "Piju" 9M2PJU
In today’s rapidly evolving tech world, the need for fast and efficient data management is more critical than ever. One name that frequently stands out in the NoSQL database world is Redis. Since its introduction in 2009, Redis has become a go-to choice for real-time applications that require exceptional speed and flexibility in handling data.
In this article, we’ll explore the history of Redis, how it’s used, and the benefits it offers to various modern applications.
The History of Redis: Origins and Evolution
Redis, which stands for Remote Dictionary Server, was developed by Salvatore Sanfilippo in 2009. Initially launched as an open-source project to address scalability issues faced by large-scale systems, Redis quickly gained popularity among developers for its ability to process data at lightning speeds.
Redis operates as an in-memory database, meaning it stores all data in RAM rather than on disk. This design enables Redis to deliver significantly faster performance compared to traditional databases, making it ideal for applications that demand real-time speed.
How is Redis Used?
One of the primary reasons Redis is so popular is its flexibility, allowing it to be used in various scenarios. Here are some real-world examples of how Redis is utilized:
- Caching
Redis is well-known for its use in caching due to its speed. By storing data in memory, Redis drastically reduces the time it takes to retrieve data. This is especially useful in web applications where users need instant access to information such as previously loaded pages, images, or API data. - Session Management
Many large platforms use Redis to store user session information. When users log into a system, Redis can store their session data in memory, ensuring quick access. This is crucial for maintaining a smooth user experience without delays. - Real-Time Analytics
In a data-driven world, companies need instant analytics to make informed decisions. Redis enables companies to process and analyze data in real time, such as tracking user behavior on websites, monitoring IoT devices, or analyzing financial transactions as they occur. - Message Queuing
Redis is also widely used for message queuing via its Pub/Sub (Publisher/Subscriber) feature. This is particularly helpful in systems where real-time communication between services or applications is required, such as notification systems or instant messaging services.
The Benefits of Redis: What Makes It Great?
Incredible Speed
Redis stands out because of its speed. As an in-memory database, Redis delivers sub-millisecond response times, making it one of the fastest technologies available for data management. This is why it is often the preferred choice for real-time applications.
Versatile Data Structures
Another feature that sets Redis apart is its support for various data structures like strings, lists, sets, and hashes. This versatility allows developers to use Redis in a wide range of scenarios, from storing user information to managing complex data in e-commerce systems.
Persistence Options
Even though Redis stores data in memory, it also offers persistence options, allowing users to periodically save data to disk. This provides an added layer of security in case of system failures, ensuring that data is backed up and recoverable.
Easy Scalability
Redis is easily scalable, whether vertically (by adding more RAM) or horizontally (by adding more Redis servers). This is essential for growing applications where the need to process more data increases over time.
Conclusion
Redis has proven itself to be one of the most powerful tools in modern data management. Its incredible speed, support for multiple data types, and scalability make it the top choice for real-time applications. Whether you’re a developer building web apps or a company looking to process real-time analytics, Redis is a technology worth exploring.
There you have it—a brief guide to Redis and the benefits it brings. This technology not only accelerates application performance but also provides a flexible and reliable solution for managing data at scale.
The post Redis: The Powerhouse Behind Modern Databases appeared first on 9M2PJU - Ham Radio, Fun Facts, Open Source Software, Tech Insights, Product Reviews by 9M2PJU.
FreeBSD vs. Ubuntu: A Comparison
Faizul "Piju" 9M2PJU
When it comes to choosing an operating system for your projects, two names often come up: FreeBSD and Ubuntu. Both have unique strengths and characteristics that make them suitable for different tasks. In this post, we’ll dive deep into the differences and similarities between these two powerful systems, helping you determine which one is the best fit for your needs.
Overview of FreeBSD and Ubuntu
FreeBSD
FreeBSD is an operating system that is derived from the Berkeley Software Distribution (BSD). Known for its performance and advanced networking features, FreeBSD provides a robust environment ideal for servers, embedded systems, and networking applications. The entire operating system, from the kernel to the userland tools, is developed from a single source, which helps ensure consistency and stability.
Ubuntu
Ubuntu is a popular Linux distribution based on Debian. It is widely used for both desktop and server environments due to its user-friendliness and extensive software repositories. Ubuntu emphasizes ease of use and regular updates, making it a favorite among beginners and experienced users alike.
Key Comparisons
1. System Base
- FreeBSD: The entire OS is developed from a single source, providing a consistent and cohesive experience. This unified approach allows for seamless integration between the kernel and userland tools.
- Ubuntu: As a Linux-based system, Ubuntu relies on the Debian base. While it offers a rich ecosystem of software, the diversity of packages can sometimes lead to compatibility issues.
2. Performance & Efficiency
- FreeBSD: Renowned for its lightweight and minimal design, FreeBSD excels in server environments where performance is critical. It manages system resources efficiently, making it ideal for high-traffic applications.
- Ubuntu: While Ubuntu performs well in most situations, its default installation comes with a variety of services and applications that can consume more system resources than necessary.
3. Software Availability
- FreeBSD: With its Ports Collection and package management system, FreeBSD offers access to over 40,000 software options. However, it may lack some of the more niche applications available on Linux.
- Ubuntu: As one of the most popular Linux distributions, Ubuntu boasts extensive software repositories, providing compatibility with nearly all Linux applications. This makes it a go-to choice for developers and users looking for variety.
4. Security
- FreeBSD: Security is a core focus of FreeBSD. It features built-in security mechanisms such as jails (which provide a form of lightweight virtualization) and a strong emphasis on minimizing vulnerabilities.
- Ubuntu: While Ubuntu is secure and receives regular updates, its wider range of installed software can lead to a larger attack surface. However, it also offers tools like AppArmor for enhanced security.
5. Community & Support
- FreeBSD: The FreeBSD community may be smaller, but it is dedicated and knowledgeable. Comprehensive documentation is available, ensuring users have access to the resources they need.
- Ubuntu: Ubuntu has a large and active community, along with professional support available through Canonical. The extensive community means users can find help quickly, whether through forums or official channels.
6. Use Cases
- FreeBSD: Ideal for servers, network appliances, and scenarios where stability and performance are paramount. Its strong networking capabilities make it a popular choice for firewalls and routers.
- Ubuntu: Excellent for desktop use, development environments, and general-purpose servers. Its ease of use makes it particularly appealing for users who are new to Linux.
Conclusion
Choosing between FreeBSD and Ubuntu ultimately comes down to your specific needs and goals. If you’re looking for an operating system that excels in performance, security, and stability, especially in server or networking environments, FreeBSD is an excellent choice. On the other hand, if you prefer a user-friendly interface with a wide array of applications for both desktop and server use, Ubuntu may be the way to go.
Both systems have their strengths, and understanding them can help you make an informed decision. Whichever you choose, you’ll be working with powerful tools that are widely respected in the tech community. Happy computing!
The post FreeBSD vs. Ubuntu: A Comparison appeared first on 9M2PJU - Ham Radio, Fun Facts, Open Source Software, Tech Insights, Product Reviews by 9M2PJU.
October 10, 2024
Xubuntu 24.10 Released
Sean Davis
Xubuntu 24.10, "Oracular Oriole," is now available, featuring many updated applications from Xfce (4.18 and 4.19), GNOME (46 and 47), and MATE (1.26).
The post Xubuntu 24.10 Released appeared first on Sean Davis.
Xubuntu 24.10 released!
Xubuntu
The Xubuntu team is happy to announce the immediate release of Xubuntu 24.10.
Xubuntu 24.10, codenamed Oracular Oriole, is a regular release and will be supported for 9 months, until July 2025.
Xubuntu 24.10 features the latest updates from Xfce 4.19, GNOME 47, and MATE 1.26. For Xfce enthusiasts, you’ll appreciate the new features and improved hardware support found in Xfce 4.19. Xfce 4.19 is the development series for the next release, Xfce 4.20, due later this year. As pre-release software, you may encounter more bugs than usual. Users seeking a stable, well-supported environment should opt for Xubuntu 24.04 “Noble Numbat” instead.
The final release images for Xubuntu Desktop and Xubuntu Minimal are available as torrents and direct downloads from xubuntu.org/download/.
As the main server might be busy in the first few days after the release, we recommend using the torrents if possible.
We’d like to thank everybody who contributed to this release of Xubuntu!
Highlights and Known Issues
Highlights
- Xfce 4.19 is included as a development preview of the upcoming Xfce 4.20. Among several new features, it features early Wayland support and improved scaling.
- GNOME 47 apps, including Disk Usage Analyzer (baobab) and Sudoku (gnome-sudoku), include a refreshed appearance and usability improvements
Known Issues
- The shutdown prompt may not be displayed at the end of the installation. Instead you might just see a Xubuntu logo, a black screen with an underscore in the upper left hand corner, or just a black screen. Press Enter and the system will reboot into the installed environment. (LP: #1944519)
- Xorg crashes and the user is logged out after logging in or switching users on some virtual machines, including GNOME Boxes. (LP: #1861609)
- You may experience choppy audio or poor system performance while playing audio, but only in some virtual machines (observed in VMware and VirtualBox)
- OEM installation options are not currently supported or available, but will be included for Xubuntu 24.04.1
For more obscure known issues, information on affecting bugs, bug fixes, and a list of new package versions, please refer to the Xubuntu Release Notes.
The main Ubuntu Release Notes cover many of the other packages we carry and more generic issues.
Support
For support with the release, navigate to Help & Support for a complete list of methods to get help.
Ubuntu Studio 24.10 Released
Ubuntu Studio
The Ubuntu Studio team is pleased to announce the release of Ubuntu Studio 24.10 code-named “Oracular Oriole”. This marks Ubuntu Studio’s 35th release. This release is a Regular release and as such, it is supported for 9 months, until July 2025.
Since it’s just out, you may experience some issues, so you might want to wait a bit before upgrading. Please see the release notes for a more complete list of changes and known issues. Listed here are some of the major highlights.
You can download Ubuntu Studio 24.10 from our download page.
Special Notes
The Ubuntu Studio 24.10 disk image (ISO) exceeds 4 GB and cannot be downloaded to some file systems such as FAT32 and may not be readable when burned to a standard DVD. For this reason, we recommend downloading to a compatible file system. When creating a boot medium, we recommend creating a bootable USB stick with the ISO image or burning to a Dual-Layer DVD.
Minimum installation media requirements: Dual-Layer DVD or 8GB USB drive.
Images can be obtained from this link: https://cdimage.ubuntu.com/ubuntustudio/releases/24.10/release/
Full updated information, including Upgrade Instructions, are available in the Release Notes.
Upgrades from 24.04 LTS should be enabled within a month after release, so we appreciate your patience.
New This Release
Minimal Installation
We have now implemented minimal installations in the system installer. This will let you install a minimal desktop to get going and then install what you need via Ubuntu Studio Installer. This will make a faster installation process and lets you customize what you need for your personal Studio.
Unfortunately, at least for the time being, we also had to get rid of the default shortcuts in the panel since it would cause an error when loading without the applications being installed. A solution for this is coming in 25.04.
Generic Kernel
The Generic Ubuntu Kernel is now fully capable of low-latency workloads. As such, with this release, we have switched from the LowLatency Kernel to the Generic Kernel with the boot options to enable the low-latency configuration enabled by default.
These options can be changed via Ubuntu Studio Audio Configuration and customized depending on your use-case and your workload. If you don’t need the low-latency and wish to have a computer that is more energy-efficient, you may wish to turn off all three options. The choice is yours.
Plasma 6
Ubuntu Studio, in cooperation with Kubuntu, switched to Plasma 6 this cycle. This switch was not without issues, so we expect many of the issues to be Plasma 6 related, especially when it comes to the default configuration and theming.
New Look
Ubuntu Studio had been using the same theming, “Materia” (except for the 22.04 LTS release which was a re-colored Breeze theme) since 19.04. However, Materia has gone dead upstream. To stay consistent, we found a fork called “Orchis” which seems to match closely and have switched to that.
As you can see from the screenshot, it has more vivid colors, round corners, and a more modern look. We hope you enjoy it. We are aware of a bug involving a dark bar under windows which may be an issue, but sometimes switching the window decorations to another variation of the theme is a solution.
PipeWire 1.2.4
This release contains PipeWire 1.2. With PipeWire 1.2, FireWire devices requiring FFADO are supported. Do note that the Ubuntu Studio team does not have any FireWire devices and could not test this.
PipeWire’s JACK compatibility is configured to use out-of-the-box and is zero-latency internally. System latency is configurable via Ubuntu Studio Audio Configuration.
However, if you would rather use straight JACK 2 instead, that’s also possible. Ubuntu Studio Audio Configuration can disable and enable PipeWire’s JACK compatibility on-the-fly. From there, you can simply use JACK via QJackCtl.
Complete Deprecation of PulseAudio/JACK setup/Studio Controls
Due to the maturity of PipeWire, the traditional PulseAudio/JACK setup, where JACK would be started/stopped by Studio Controls and bridged to PulseAudio, is now fully deprecated and the option is not offered anymore. This configuration is no longer installable via Ubuntu Studio Audio Configuration. Studio Controls may return someday as a PipeWire fine-tuning solution, but for now it is unsupported by the developer.
Ardour 8.6
While this does not represent the latest release of Ardour, Ardour 8.6 is a great release. If you would like the latest release, we highly recommend purchasing one-time or subscribing to Ardour directly from the developers to help support this wonderful application.
To help support Ardour’s funding, you may obtain later versions directly from ardour.org. To do so, please one-time purchase or subscribe to Ardour from their website. If you wish to get later versions of Ardour from us, you will have to wait until the next regular release of Ubuntu Studio, due in April 2025.
Ubuntu Studio Audio Configuration
Ubuntu Studio Audio Configuration’s Dummy Audio Device now also has a much-requested Dummy Audio Input.
Additionally as described above, Ubuntu Studio Audio Configuration has an option to configure the default boot parameters that are commonly used to enable the low-latency capabilities of the Linux kernel used in Ubuntu. For more information about that, see the Ubuntu Studio Audio Configuration page.
We’re back on Matrix
You’ll notice that the menu links to our support chat and on our website will now take you to a Matrix chat. This is due to the Ubuntu community carving its own space within the Matrix federation.
However, this is not only a support chat. This is also a creativity discussion chat. You can pass ideas to each other and you’re welcome to it if the topic remains within those confines. However, if a moderator or admin warns you that you’re getting off-topic (or the intention for the chat room), please heed the warning.
This is a persistent connection, meaning if you close the window (or chat), it won’t lose your place as you may only need to sign back in to resume the chat.
Frequently Asked Questions
Q: Does Ubuntu Studio contain snaps?
A: Yes. Mozilla’s distribution agreement with Canonical changed, and Ubuntu was forced to no longer distribute Firefox in a native .deb package. We have found that, after numerous improvements, Firefox now performs just as well as the native .deb package did.
Thunderbird also became a snap so that the maintainers can get security patches delivered faster.
Additionally, Freeshow is an Electron-based application. Electron-based applications cannot be packaged in the Ubuntu repositories in that they cannot be packaged in a traditional Debian source package. While such apps do have a build system to create a .deb binary package, it circumvents the source package build system in Launchpad, which is required when packaging for Ubuntu. However, Electron apps also have a facility for creating snaps, which can be uploaded and included. Therefore, for Freeshow to be included in Ubuntu Studio, it had to be packaged as a snap.
We have additional snaps that are Ubuntu-specific, such as the Firmware Updater and the Security Center. Contrary to popular myth, Ubuntu does not have any plans to switch all packages to snaps, nor do we.
Q: Will you make an ISO with {my favorite desktop environment}?
A: To do so would require creating an entirely new flavor of Ubuntu, which would require going through the Official Ubuntu Flavor application process. Since we’re completely volunteer-run, we don’t have the time or resources to do this. Instead, we recommend you download the official flavor for the desktop environment of your choice and use Ubuntu Studio Installer to get Ubuntu Studio – which does *not* convert that flavor to Ubuntu Studio but adds its benefits.
Q: What if I don’t want all these packages installed on my machine?
A: Simply use the Ubuntu Studio Installer to remove the features of Ubuntu Studio you don’t want or need!
Get Involved!
A wonderful way to contribute is to get involved with the project directly! We’re always looking for new volunteers to help with packaging, documentation, tutorials, user support, and MORE! Check out all the ways you can contribute!
Our project leader, Erich Eickmeyer, is now working on Ubuntu Studio at least part-time, and is hoping that the users of Ubuntu Studio can give enough to generate a monthly part-time income. We’re not there, but if every Ubuntu Studio user donated monthly, we’d be there! Your donations are appreciated! If other distributions can do it, surely we can! See the sidebar for ways to give!
Special Thanks
Huge special thanks for this release go to:
- Eylul Dogruel: Artwork, Graphics Design
- Ross Gammon: Upstream Debian Developer, Testing, Email Support
- Sebastien Ramacher: Upstream Debian Developer
- Dennis Braun: Upstream Debian Developer
- Rik Mills: Kubuntu Council Member, help with Plasma desktop
- Scarlett Moore: Kubuntu Project Lead, help with Plasma desktop
- Cristian Delgado: Translations for Ubuntu Studio Menu
- Dan Bungert: Subiquity, seed fixes
- Len Ovens: Testing, insight
- Wim Taymans: Creator of PipeWire
- Mauro Gaspari: Tutorials, Promotion, and Documentation, Testing, keeping Erich sane
- Krytarik Raido: IRC Moderator, Mailing List Moderator
- Erich Eickmeyer: Project Leader, Packaging, Development, Direction, Treasurer
Kubuntu 24.10 Oracular Oriole Released
Kubuntu General News
The Kubuntu Team is happy to announce that Kubuntu 24.10 has been released, featuring the new and beautiful KDE Plasma 6.1 simple by default, powerful when needed.
Codenamed “Oracular Oriole”, Kubuntu 24.10 continues our tradition of giving you Friendly Computing by integrating the latest and greatest open source technologies into a high-quality, easy-to-use Linux distribution.
Under the hood, there have been updates to many core packages, including a new 6.11 based kernel, KDE Frameworks 5.116 and 6.6.0, KDE Plasma 6.1 and many updated KDE gear applications.
Kubuntu has seen many updates for other applications, both in our default install, and installable from the Ubuntu archive.
Applications for core day-to-day usage are included and updated, such as Firefox, and LibreOffice.
For a list of other application updates, and known bugs be sure to read our release notes.
Wayland as default Plasma session.
The Plasma wayland session is now the default option in sddm (display manager login screen). An X11 session can be selected instead if desired. The last used session type will be remembered, so you do not have to switch type on each login.
Download Kubuntu 24.10, or learn how to upgrade from 24.04 LTS.
Note: For upgrades from 24.04, there may a delay of a few hours to days between the official release announcements and the Ubuntu Release Team enabling upgrades.
Lubuntu 24.10 (Oracular Oriole) Released!
Lubuntu Blog
October 08, 2024
Ubuntu MATE 24.10 Release Notes
Ubuntu MATE
Ubuntu MATE 24.10 is more of what you like, stable MATE Desktop on top of current Ubuntu. Read on to learn more 👓️
Ubuntu MATE 24.10
Thank you! 🙇
My sincere thanks to everyone who has played an active role in improving Ubuntu MATE for this release 👏 I’d like to acknowledge the close collaboration with the Ubuntu Foundations team and the Ubuntu flavour teams, in particular Erich Eickmeyer who pushed critical fixes while I was travelling. Thank you! 💚
What changed since the Ubuntu MATE 24.04 LTS?
Here are the highlights of what’s changed since the release of Ubuntu MATE 24.04
- Ships stable MATE Desktop 1.26.2 with a handful of bug fixes 🐛
- Switched back to Slick Greeter (replacing Arctica Greeter) due to race-condition in the boot process which results the display manager failing to initialise.
- Returning to Slick Greeter reintroduces the ability to easily configure the login screen via a graphical application, something users have been requesting be re-instated 👍
- Ubuntu MATE 24.10 .iso 📀 is now 3.3GB 🤏 Down from 4.1GB in the 24.04 LTS release.
- This is thanks to some fixes in the installer that no longer require as many packages in the live-seed.
Login Window
What didn’t change since the Ubuntu MATE 24.04 LTS?
If you follow upstream MATE Desktop development, then you’ll have noticed that Ubuntu MATE 24.10 doesn’t ship with the recently released MATE Desktop 1.28 🧉
I have prepared packaging for MATE Desktop 1.28, along with the associated components but encountered some bugs and regressions 🐞 I wasn’t able to get things to a standard I’m happy to ship be default, so it is tried and true MATE 1.26.2 one last time 🪨
Major Applications
Accompanying MATE Desktop 1.26.2 🧉 and Linux 6.11 🐧 are Firefox 131 🔥🦊, Celluloid 0.27 🎥, Evolution 3.54 📧, LibreOffice 24.8.2 📚
See the Ubuntu 24.10 Release Notes for details of all the changes and improvements that Ubuntu MATE benefits from.
Download Ubuntu MATE 24.10
Ubuntu MATE 24.10 (Oracular Oriole) is available for PC/Mac users.
DownloadUpgrading to Ubuntu MATE 24.10
The upgrade process to Ubuntu MATE 24.10 is the same as Ubuntu.
There are no offline upgrade options for Ubuntu MATE. Please ensure you have network connectivity to one of the official mirrors or to a locally accessible mirror and follow the instructions above.
October 07, 2024
Ubuntu Budgie 24.10 Released
David Mohammed
Ubuntu Budgie 24.10 (Oracular Oriole) is a Standard Release with 9 months of support by your distro maintainers and Canonical, from Oct 2024 to July 2025.. These release notes showcase the key takeaways for 24.04 upgraders to 24.10. In these release notes the areas covered are: The key focus for the team for this cycle has been the conversion of our distro to a Wayland based distro.
September 29, 2024
A networking guide for Incus
Simos Xenitellis
Incus is a hypervisor/manager for virtual machines and application/system containers. Get community support here.
A virtual machine (VM) is an instance of an operating system that runs on a computer, along with the main operating system. A virtual machine uses hardware virtualization features for the separation from the main operating system.
A system container is an instance of an operating system that also runs on a computer, along with the main operating system. A system container, instead, uses security primitives of the Linux kernel for the separation from the main operating system. The system container follows the lifecycle of a computer system. You can think of system containers as software virtual machines.
An application container is a container that has an application or service. It follows the lifecycle of the application instead of a system. That is, here you start and stop the application instead of booting and shutting down a system. Incus supports Open Container Initiative (OCI) images such as Docker images. When Incus launches an OCI image, it uses its own runtime, not Docker’s. That is, Incus consumes images from any OCI image repositories.
In virtual machines and system/application containers we can attach virtual networking devices, either
- none, (i.e. an instance without networking)
- one or, (i.e. most common and simple case)
- more than one.
In addition to the virtual networking devices, we can also attach real hardware networking devices. Those devices can be taken away from the host and get pushed into a virtual machine or system container.
You may use a combination of those networking devices in the same instance. It is left as an exercise to the reader to explore that road. In these tutorials we are look at one at most networking device per instance.
There will be attempts to generalize and explain in practical terms. If I get something wrong, please correct me in the comments so that it gets fixed and we all learn something new. Note that I will be editing this content along the way, adding material, troubleshooting cases, etc.
In this post we are listing tutorials of the different Incus devices of type nic (network interface controller). Whatever we write in this post and the linked tutorials, are covered in that documentation URL!
The list of tutorials per networking:
bridge
(the default, the local network bridge), it’s in this post below.bridged
, (pending)macvlan
, (pending)- none,
physical
,ipvlan
,routed
,
The setup
When demonstrating these network configurations, we will be using an Incus VM. When learning, try there in your Incus VM before applying on your host or your server.
We launch an Incus VM, called tutorial, with Ubuntu 24.04 LTS, then get a shell with the default non-root account ubuntu. I am impatient and I am typing repeatedly the incus exec
command to get a shell. The VM takes a few moments to boot up, and I get interested error messages until the VM is actually running. Not really relevant to this tutorial but you will get educated at every opportunity.
$ incus launch images:ubuntu/24.04/cloud tutorial --vm
Launching tutorial
$ incus exec tutorial -- su -l ubuntu
Error: VM agent isn't currently running
$ incus exec tutorial -- su -l ubuntu
su: user ubuntu does not exist or the user entry does not contain all the required fields
$ incus exec tutorial -- su -l ubuntu
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
ubuntu@tutorial:~$
We got a shell in the VM. Then, install Incus which is available in the default repositories of Ubuntu 24.04 LTS. Also, we install zfsutils-linux
, which are the client utilities to use ZFS in Incus. We are advised to add our non-root account to the incus-admin
group in order to have access to Incus. Without that, we would have to use sudo
all the time. When you add a user to a group, you need to logout then log in again for the change to take effect. And this is what we do (unless you know about newgrp
).
ubuntu@tutorial:~$ sudo apt install -y incus zfsutils-linux
...
Creating group 'incus' with GID 989.
Creating group 'incus-admin' with GID 988.
Created symlink /etc/systemd/system/multi-user.target.wants/incus-startup.service → /usr/lib/systemd/system/incus-startup.service.
Created symlink /etc/systemd/system/sockets.target.wants/incus-user.socket → /usr/lib/systemd/system/incus-user.socket.
Created symlink /etc/systemd/system/sockets.target.wants/incus.socket → /usr/lib/systemd/system/incus.socket.
incus.service is a disabled or a static unit, not starting it.
incus-user.service is a disabled or a static unit, not starting it.
Incus has been installed. You must run `sudo incus admin init` to
perform the initial configuration of Incus.
Be sure to add user(s) to either the 'incus-admin' group for full
administrative access or the 'incus' group for restricted access,
then have them logout and back in to properly setup their access.
...
ubuntu@tutorial:~$ sudo usermod -a -G incus-admin ubuntu
ubuntu@tutorial:~$ logout
$ incus exec tutorial -- su -l ubuntu
ubuntu@tutorial:~$
Now we initialize Incus with sudo incus admin init
.
Default Incus networking
When you install and setup Incus with incus admin init
, you are prompted whether you want to create a local network bridge. We press Enter to all prompts, which means that we accept all the defaults that are presented to us. The last question is whether to show the initialization configuration. If you missed it, you can get it after the fact by running incus admin init --dump
(dumps the configuration).
ubuntu@tutorial:~$ incus admin init
Would you like to use clustering? (yes/no) [default=no]:
Do you want to configure a new storage pool? (yes/no) [default=yes]:
Name of the new storage pool [default=default]:
Name of the storage backend to use (zfs, dir) [default=zfs]:
Create a new ZFS pool? (yes/no) [default=yes]:
Would you like to use an existing empty block device (e.g. a disk or partition)? (yes/no) [default=no]:
Size in GiB of the new loop device (1GiB minimum) [default=5GiB]:
Would you like to create a new local network bridge? (yes/no) [default=yes]:
What should the new bridge be called? [default=incusbr0]:
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
Would you like the server to be available over the network? (yes/no) [default=no]:
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]:
Would you like a YAML "init" preseed to be printed? (yes/no) [default=no]: yes
config: {}
networks:
- config:
ipv4.address: auto
ipv6.address: auto
description: ""
name: incusbr0
type: ""
project: default
storage_pools:
- config:
size: 5GiB
description: ""
name: default
driver: zfs
profiles:
- config: {}
description: ""
devices:
eth0:
name: eth0
network: incusbr0
type: nic
root:
path: /
pool: default
type: disk
name: default
projects: []
cluster: null
ubuntu@tutorial:~$
If you accept the defaults (i.e. press Enter in each) or type them explicitly, you get a local bridge named incusbr0
that is managed by Incus, and gives private IPv4 and IPv6 IP addresses to your newly created instances.
Let’s see them in practice in your Incus installation. You have configured Incus and Incus created a default profile, called default, for you. This profile is applied by default to all newly created instances and has the networking configuration in there. In that profile there are two devices, and one of them is the networking device. In Incus the device is called eth0
(in pink color), and in the instance it will be shown as eth0
(green color). On the host, the bridge will appear with the name incusbr0
. It’s a networking type, hence of type nic
.
ubuntu@tutorial:~$ incus profile list
+---------+-----------------------+---------+
| NAME | DESCRIPTION | USED BY |
+---------+-----------------------+---------+
| default | Default Incus profile | 0 |
+---------+-----------------------+---------+
ubuntu@tutorial:~$ incus profile show default
config: {}
description: Default Incus profile
devices:
eth0:
name: eth0
network: incusbr0
type: nic
root:
path: /
pool: default
type: disk
name: default
used_by: []
ubuntu@tutorial:~$
incusbr0
was created by Incus. Let’s see details through the incus network
commands. We first list the network interfaces and then we show the incusbr0
network interface. incusbr0
is a managed network interface (in pink below), and it’s managed by Incus. Incus takes care of the networking and provides DHCP services, and access to the upstream network (i.e. the Internet). incusbr0
is a network bridge (in blue). An instance that requires network configuration from incusbr0
, will get an IP address from the range 10.180.234.1-254
(in orange). Network Address Translation (NAT) is enabled (also in orange), which means there is access to the upstream network, and likely the Internet.
ubuntu@tutorial:~$ incus network list
+----------+----------+---------+-----------------+---------+---------+
| NAME | TYPE | MANAGED | IPV4 | USED BY | STATE |
+----------+----------+---------+-----------------+---------+---------+
| enp5s0 | physical | NO | | 0 | |
+----------+----------+---------+-----------------+---------+---------+
| incusbr0 | bridge | YES | 10.180.234.1/24 | 1 | CREATED |
+----------+----------+---------+-----------------+---------+---------+
ubuntu@tutorial:~$ incus network show incusbr0
config:
ipv4.address: 10.180.234.1/24
ipv4.nat: "true"
ipv6.address: fd42:7:7dfe:75cf::1/64
ipv6.nat: "true"
description: ""
name: incusbr0
type: bridge
used_by:
- /1.0/profiles/default
managed: true
status: Created
locations:
- none
ubuntu@tutorial:~$
Let’s launch a container and test these out. The instance got an IP address (in orange) that is within the range of the network bridge above.
ubuntu@tutorial:~$ incus launch images:alpine/edge/cloud myalpine
Launching myalpine
ubuntu@tutorial:~$ incus list -c ns4t
+----------+---------+----------------------+-----------+
| NAME | STATE | IPV4 | TYPE |
+----------+---------+----------------------+-----------+
| myalpine | RUNNING | 10.180.234.24 (eth0) | CONTAINER |
+----------+---------+----------------------+-----------+
ubuntu@tutorial:~$
The IP address is OK but could it look better? It’s private anyway, and we can select anything from the range 10.x.y.z
. Let’s change it so that it uses instead 10.10.10.1-254
. We set the configuration of incusbr0
for ipv4.address
(see earlier) to a new value, 10.10.10.1/24
. Each number separated by commas is 8 bits in length, and /24
means that the first 3 * 8 = 24 bits should stay the same. We make the change, but the instance still has the old IP address. We restart the instance, and it automatically gets the new IP address from the new range.
ubuntu@tutorial:~$ incus network set incusbr0 ipv4.address=10.10.10.1/24
ubuntu@tutorial:~$ incus list -c ns4t
+----------+---------+----------------------+-----------+
| NAME | STATE | IPV4 | TYPE |
+----------+---------+----------------------+-----------+
| myalpine | RUNNING | 10.180.234.24 (eth0) | CONTAINER |
+----------+---------+----------------------+-----------+
ubuntu@tutorial:~$ incus restart myalpine
ubuntu@tutorial:~$ incus list -c ns4t
+----------+---------+--------------------+-----------+
| NAME | STATE | IPV4 | TYPE |
+----------+---------+--------------------+-----------+
| myalpine | RUNNING | 10.10.10.24 (eth0) | CONTAINER |
+----------+---------+--------------------+-----------+
ubuntu@tutorial:~$
We have created incusbr0
. Are we allowed to create another private bridge? Sure we are. We will call it incusbr1
, and also we disable IPv6 networking. IPv6 addresses are too wide and mess up the formatting on my blog. If you notice earlier, there were no IPv6 addresses although IPv6 was configured on incusbr0
. I cheated and removed the IPv6 addresses in some command outputs.
ubuntu@tutorial:~$ incus network create incusbr1 ipv4.address=10.10.20.1/24 ipv6.address=none
Network incusbr1 created
ubuntu@tutorial:~$ incus network show incusbr1
config:
ipv4.address: 10.10.20.1/24
ipv6.address: none
description: ""
name: incusbr1
type: bridge
used_by: []
managed: true
status: Created
locations:
- none
ubuntu@tutorial:~$
We have created incusbr1
. Can we now launch an instance onto that private bridge? We launch the instance called myalpine1 and we used the incus launch
parameter --network incusbr1
to specify a different network than the default network in the default Incus profile. We verify below that myalpine1 is served by incusbr1 (in green).
ubuntu@tutorial:~$ incus launch images:alpine/edge/cloud myalpine1 --network incusbr1
Launching myalpine1
ubuntu@tutorial:~$ incus list -c ns4t
+-----------+---------+--------------------+-----------+
| NAME | STATE | IPV4 | TYPE |
+-----------+---------+--------------------+-----------+
| myalpine | RUNNING | 10.10.10.24 (eth0) | CONTAINER |
+-----------+---------+--------------------+-----------+
| myalpine1 | RUNNING | 10.10.20.85 (eth0) | CONTAINER |
+-----------+---------+--------------------+-----------+
ubuntu@tutorial:~$ incus network show incusbr1
config:
ipv4.address: 10.10.20.1/24
ipv6.address: none
description: ""
name: incusbr1
type: bridge
used_by:
- /1.0/instances/myalpine1
managed: true
status: Created
locations:
- none
ubuntu@tutorial:~$
Technical details
The instances that use the Incus private bridge have access to the Internet. How is this achieved? It’s achieve with either iptables
or nftables
rules. In recent versions of Linux distributions, you would be using nftables
by default (command: nft
, no relation to NFTs). To view the firewall ruleset that were created by Incus, run sudo nft list ruleset
. Here is my ruleset and should be similar to yours. There is one table for Incus and four chains. A persistent, a forward, an in and an out. More at the documentation site at nftables.
ubuntu@tutorial:~$ sudo nft list ruleset
table inet incus {
chain pstrt.incusbr0 {
type nat hook postrouting priority srcnat; policy accept;
ip saddr 10.57.39.0/24 ip daddr != 10.57.39.0/24 masquerade
ip6 saddr fd42:e7b:739c:7117::/64 ip6 daddr != fd42:e7b:739c:7117::/64 masquerade
}
chain fwd.incusbr0 {
type filter hook forward priority filter; policy accept;
ip version 4 oifname "incusbr0" accept
ip version 4 iifname "incusbr0" accept
ip6 version 6 oifname "incusbr0" accept
ip6 version 6 iifname "incusbr0" accept
}
chain in.incusbr0 {
type filter hook input priority filter; policy accept;
iifname "incusbr0" tcp dport 53 accept
iifname "incusbr0" udp dport 53 accept
iifname "incusbr0" icmp type { destination-unreachable, time-exceeded, parameter-problem } accept
iifname "incusbr0" udp dport 67 accept
iifname "incusbr0" icmpv6 type { destination-unreachable, packet-too-big, time-exceeded, parameter-problem, nd-router-solicit, nd-neighbor-solicit, nd-neighbor-advert, mld2-listener-report } accept
iifname "incusbr0" udp dport 547 accept
}
chain out.incusbr0 {
type filter hook output priority filter; policy accept;
oifname "incusbr0" tcp sport 53 accept
oifname "incusbr0" udp sport 53 accept
oifname "incusbr0" icmp type { destination-unreachable, time-exceeded, parameter-problem } accept
oifname "incusbr0" udp sport 67 accept
oifname "incusbr0" icmpv6 type { destination-unreachable, packet-too-big, time-exceeded, parameter-problem, echo-request, nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert, mld2-listener-report } accept
oifname "incusbr0" udp sport 547 accept
}
}
ubuntu@tutorial:~$
Future considerations
September 21, 2024
Kubuntu Oracular Oriole (24.10) Beta released
Kubuntu General News
The beta of Kubuntu Oracular Oriole (to become 24.10 in October) has now been released, and is available for download.
This milestone features images for Kubuntu and other Ubuntu flavours.
Pre-releases of Kubuntu Mantic Minotaur are not recommended for:
- Anyone needing a stable system
- Regular users who are not aware of pre-release issues
- Anyone in a production environment with data or workflows that need to be reliable
They are, however, recommended for:
- Regular users who want to help us test by finding, reporting, and/or fixing bugs
- Kubuntu, KDE, and Qt developers
- Other Ubuntu flavour developers
The Beta includes some software updates that are ready for broader testing. However, it is an early set of images, so you should expect some bugs.
We STRONGLY advise testers to read the Kubuntu 24.10 Beta release notes before installing, and in particular the section on ‘Known issues‘.
You can also find more information about the entire 24.10 release (base, kernel, graphics etc) in the main Ubuntu Beta release notes and announcement.
To enable Flatpaks in KDE’s Discover in Kubuntu 24.10, run this command:
sudo apt install flatpak plasma-discover-backend-flatpak
To enable the largest Flatpak repository, run this command:
flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
Log out and log back in (or restart) to re-initialize the XDG_DATA_DIRS variable, otherwise, newly installed Flatpak apps will not run or appear in the startup menu.
Ubuntu Studio 24.10 Beta Released
Ubuntu Studio
The Ubuntu Studio team is pleased to announce the beta release of Ubuntu Studio 24.10, codenamed “Oracular Oriole”.
While this beta is reasonably free of any showstopper installer bugs, you will find some bugs within. This image is, however, mostly representative of what you will find when Ubuntu Studio 24.10 is released on October 10, 2024.
Special Notes
The Ubuntu Studio 24.10 image (ISO) exceeds 4 GB and cannot be downloaded to some file systems such as FAT32 and may not be readable when burned to a DVD. For this reason, we recommend downloading to a compatible file system. When creating a boot medium, we recommend creating a bootable USB stick with the ISO image or burning to a Dual-Layer DVD.
Images can be obtained from this link: https://cdimage.ubuntu.com/ubuntustudio/releases/24.10/beta/
Full updated information, including Upgrade Instructions, are available in the Release Notes.
New Features This Release
- Plasma 6.1 is now the default desktop environment, an upgrade from Plasma 5.27. This may have some unknown bugs that we’re ironing out as we go along, along with theming.
- Ubuntu’s Generic Kernel is now capable of the same low latency processing as Ubuntu’s lowlatency kernel when certain boot parameters are used. Additionally, the lowlatency kernel is eventually going to be deprecated. With this in mind, we have switched to the generic kernel with the low latency boot parameters enabled by default. These boot parameters can be tweaked in Ubuntu Studio Audio Configuation.
- Minimal Install Option for new installations. This allows users to install Ubuntu Studio and customize what they need later with Ubuntu Studio Installer.
- Orchis is now our default theme, which replaces Materia, our default theme since 19.04. Materia has stopped development, so we decided to
- PipeWire continues to improve with every release and now includes FFADO support. Version 1.2.3
- Ubuntu Studio Installer‘s included Ubuntu Studio Audio Configuration utility for fine-tuning the PipeWire setup now includes the ability to create or remove a dummy audio input device. Version 1.30
- The legacy PulseAudio/JACK has been deprecated and discontinued, is no longer supported, and is no longer an option to use. Going forward, PipeWire or JACK are the only options. PipeWire’s JACK integration can be disabled from Ubuntu Studio Audio Configuration to use JACK by itself with QJackCtl, or via other means.
Major Package Upgrades
- Ardour version 8.6.0
- Qtractor version 1.1
- OBS Studio version 30.2.3
- Audacity version 3.6.1
- digiKam version 8.4.0
- Kdenlive version 24.08.1
- Krita version 5.2.3
There are many other improvements, too numerous to list here. We encourage you to look around the freely-downloadable ISO image.
Known Issues
- Due to the transition to Plasma 6 and Qt6, there may be some theming inconsistencies, especially for those upgrading. To work around these issues, reapply the default theme using System Settings and select “Orchis-dark” from Kvantum Manager.
- Some graphics cards might find the transparency in the Orchis theme difficult to work with. For that reason, you can switch to “Orchis-dark-solid” in the Kvantum Manager. Feedback is welcome, and if the transparency becomes too burdensome, we can switch to the solid theme by default.
- The new minimal install mode will not load the desktop properly with the extra icons (gimp, krita, patchance, etc.) in the top bar, so those had to be removed by default. If you find them useful, you can add them by right-clicking in the menu and clicking “Pin to Task Manager”. We apologize for the inconvenience.
Official Ubuntu Studio release notes can be found at https://ubuntustudio.org/ubuntu-studio-24-10-release-notes/
Further known issues, mostly pertaining to the desktop environment, can be found at https://wiki.ubuntu.com/OracularOriole/ReleaseNotes/Kubuntu
Additionally, the main Ubuntu release notes contain more generic issues: https://discourse.ubuntu.com/t/oracular-oriole-release-notes/44878
How You Can Help
Please test using the test cases on https://iso.qa.ubuntu.com. All you need is a Launchpad account to get started.
Additionally, we need financial contributions. Our project lead, Erich Eickmeyer, is working long hours on this project and trying to generate a part-time income. Go here to see how you can contribute financially (options are also in the sidebar).
Frequently Asked Questions
Q: Does Ubuntu Studio contain snaps?
A: Yes. Mozilla’s distribution agreement with Canonical changed, and Ubuntu was forced to no longer distribute Firefox in a native .deb package. We have found that, after numerous improvements, Firefox now performs just as well as the native .deb package did.
Thunderbird is also a snap this cycle in order for the maintainers to get security patches delivered faster.
Additionally, Freeshow is an Electron-based application. Electron-based applications cannot be packaged in the Ubuntu repositories in that they cannot be packaged in a traditional Debian source package. While such apps do have a build system to create a .deb binary package, it circumvents the source package build system in Launchpad, which is required when packaging for Ubuntu. However, Electron apps also have a facility for creating snaps, which can be uploaded and included. Therefore, for Freeshow to be included in Ubuntu Studio, it had to be packaged as a snap.
Also, to keep theming consistent, all included themes are snapped in addition to the included .deb versions so that snaps stay consistent with out themes.
We are working with Canonical to make sure that the quality of snaps goes up with each release, so we please ask that you give snaps a chance instead of writing them off completely.
Q: If I install this Beta release, will I have to reinstall when the final release comes out?
A: No. If you keep it updated, your installation will automatically become the final release. However, if Audacity returns to the Ubuntu repositories before final release, then you might end-up with a double-installation of Audacity. Removal instructions of one or the other will be made available in a future post.
Q: Will you make an ISO with {my favorite desktop environment}?
A: To do so would require creating an entirely new flavor of Ubuntu, which would require going through the Official Ubuntu Flavor application process. Since we’re completely volunteer-run, we don’t have the time or resources to do this. Instead, we recommend you download the official flavor for the desktop environment of your choice and use Ubuntu Studio Installer to get Ubuntu Studio – which does *not* convert that flavor to Ubuntu Studio but adds its benefits.
Q: What if I don’t want all these packages installed on my machine?
A: We now include a minimal install option. Install using the minimal install option, then use Ubuntu Studio Installer to install what you need for your very own content creation studio.
September 17, 2024
My Chair
Benjamin Mako Hill
I realize that because I have several chairs, the phrase “my chair” is ambiguous. To reduce confusion, I will refer to the head of my academic department as “my office chair” going forward.
September 13, 2024
Where are Podcast Listener Communities
Alan Pope
Parasocial chat
On Linux Matters we have a friendly and active, public Telegram channel linked on our Contact page, along with a Discord Channel. We also have links to Mastodon, Twitter (not that we use it that much) and email.
At the time of writing there are roughly this ⬇️ number of people (plus bots, sockpuppets and duplicates) in or following each Linux Matters “official” presence:
Channel | Number |
---|---|
Telegram | 796 |
Discord | 683 |
Mastodon | 858 |
9919 |
Preponderance of chat
We chose to have a presence in lots of places, but primarily the talent presenters (Martin, Mark, and myself (and Joe)) only really hang out to chat on Telegram and Mastodon.
I originally created the Telegram channel on November 20th, 2015, when we were publishing the Ubuntu Podcast (RIP in Peace) A.K.A. Ubuntu UK Podcast. We co-opted and renamed the channel when Linux Matters launched in 2023.
Prior to the channel’s existence, we used the Ubuntu UK Local Community (LoCo) Team IRC channel on Freenode (also, RIP in Peace).
We also re-branded our existing Mastodon accounts from the old Ubuntu Podcast to Linux Matters.
We mostly continue using Telegram and Mastodon as our primary methods of communication because on the whole they’re fast, reliable, stay synced across devices, have the features we enjoy, and at least one of them isn’t run by a weird billionaire.
Other options
We link to a lot of other places at the top of the Linux Matters home page, where our listeners can chat, mostly to eachother and not us.
Being over 16, I’m not a big fan of Discord, and I know Mark doesn’t even have an account there. None of us use Twitter much anymore, either.
Periodically I ponder if we (Linux Matters) should use something other than Telegram. I know some listeners really don’t like the platform, but prefer other places like Signal, Matrix or even IRC. I know for sure some non-listeners don’t like Telegram, but I care less about their opinions.
Part of the problem is that I don’t think any of us really enjoy the other realtime chat alternatives. Both Matrix and Signal have terrible user experience, and other flaws. Which is why you don’t tend to find us hanging out in either of those places.
There are further options I haven’t even considered, like Wire, WhatsApp, and likely more I don’t even know or care about.
So we kept using Telegram over any of the above alternative options.
Pondering Posting Polls
I have repeatedly considered asking the listeners about their preferred chat platforms via our existing channels. But that seems flawed, because we use what we like, and no matter how many people prefer something else, we’re unlikely to move. Unless something strange happens 👀 .
Plus, often times, especially on decentralised platforms, the audience can be somewhat “over-enthusiastic” about their preferred way being The Way™️
over the alternatives. It won’t do us any favours to get data saying 40% report we should use Signal, 40% suggest Matrix and 20% choose XMPP, if the four of us won’t use any of them.
Pursue Podcast Palaver Proposals
So rather than ask our audience, I thought I’d see what other podcasters promote for feedback and chatter on their websites.
I picked a random set from shows I have heard of, and may have listened to, plus a few extra ones I haven’t. None of this is endorsement or approval, I wanted the facts, just the fax, ma’am.
I collated the data in a json file for some reason, then generated the tables below. I don’t know what to do with this information, but it’s a bit of data we may use if we ever decide to move away from Telegram.
Presenting Pint-Sized Payoff
The table shows some nerdy podcasts along with their primary means (as far as I can tell) of community engagement. Data was gathered manually from podcast home pages and “about” pages. I generally didn’t go into the page content for each episode. I made an exception for “Dot Social” and “Linux OTC” because there’s nothing but episodes on their home page.
It doesn’t matter for this research, I just thought it was interesting that some podcasters don’t feel the need to break out their contact details to a separate page, or make it more obvious. Perhaps they feel that listeners are likely to be viewing an episode page, or looking at a specific show metadata, so it’s better putting the contact details there.
I haven’t included YouTube, where many shows publish and discuss, in addition to a podcast feed.
I am also aware that some people exclusively, or perhaps primarily publish on YouTube (or other video platforms). Those aren’t podcasts IMNSHO.
Key to the tables below. Column names have been shorted because it’s a w i d e table. The numbers indicate how many podcasts use that communication platform.
- EM - Email address (13/18)
- MA - Mastodon account (9/18)
- TW - Twitter account (8/18)
- DS - Discord server (8/18)
- TG - Telegram channel (4/18)
- IR - IRC channel (5/18)
- DW - Discourse website (2/18)
- SK - Slack channel (3/18)
- LI - LinkedIn (2/18)
- WF - Web form (2/18)
- SG - Signal group (3/18)
- WA - WhatsApp (1/18)
- FB - FaceBook (1/18)
Linux
Show | EM | MA | TW | DS | TG | IR | DW | SK | MX | LI | WF | SG | WA | FB |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Linux Matters | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ||||||||
Ask The Hosts | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||||
Destination Linux | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||||
Linux Dev Time | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||||
Linux After Dark | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||||
Linux Unplugged | ✅ | ✅ | ✅ | ✅ | ||||||||||
This Week in Linux | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||||
Ubuntu Security Podcast | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||||
Linux OTC | ✅ | ✅ | ✅ |
Open Source Adjunct
Show | EM | MA | TW | DS | TG | IR | DW | SK | MX | LI | WF | SG | WA | FB |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2.5 Admins | ✅ | ✅ | ||||||||||||
Bad Voltage | ✅ | ✅ | ✅ | ✅ | ||||||||||
Coffee and Open Source | ✅ | |||||||||||||
Dot Social | ✅ | ✅ | ||||||||||||
Open Source Security | ✅ | ✅ | ✅ | |||||||||||
localfirst.fm | ✅ |
Other Tech
Show | EM | MA | TW | DS | TG | IR | DW | SK | MX | LI | WF | SG | WA | FB |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ATP | ✅ | ✅ | ✅ | ✅ | ||||||||||
BBC Newscast | ✅ | ✅ | ✅ | |||||||||||
The Rest is Entertainment | ✅ |
Point
Not entirely sure what to do with this data. But there it is.
Is Linux Matters going to move away from Telegram to something else? No idea.
September 12, 2024
git revert name and Akademy
Jonathan Riddell
I reverted my name back to Jonathan Riddell and have now made a new uid for my PGP key, you can get the updated one on keyserver.ubuntu.com or my contact page or my Launchpad page.
Here’s some pics from Akademy
September 11, 2024
How to recover or reconnect an Incus storage pool
Simos Xenitellis
Incus is a manager for virtual machines, system containers and application containers. Get Incus support here.
When you initially setup Incus, you create a storage pool where Incus will put in there everything. There are several options for storage pools, in this post we focus on ZFS storage pools, and those specifically that are stored on a separate block device (like /dev/sdb
).
We are dealing with two cases. One, your installation of Incus has been somehow removed but the storage pool is somewhere there intact and you want to recover by installing again Incus. Two, you want to move the disk with storage pool from one computer to another, like reconnecting the storage pool on a new server.
This type of task is quite risky if you have a lot of important data on your system. Obviously, prior to you actually doing this on an actual system, you should take backups with incus export
of your most important instances. And then, you should perform this tutorial several times so that you get the gist of recovering Incus installations. This tutorial shows you how to do a dry run of creating an Incus installation, killing it off, and then miraculously recovering it.
Prerequisites
You should have a running Incus installation.
Setting up Incus, using a block storage volume
We launch an Incus virtual machine (VM) that will act as our Incus server. We then (on the host) create a storage volume of type block. Next, we attach that block storage volume to the VM. In the VM it can be found as /dev/sdb
. Subsequently, we incus admin init
to initialize Incus, and configure Incus to use the block device /dev/sdb
when creating the storage pool. When we run incus admin init
, we press Enter when we want to accept the default value.
$ incus launch images:ubuntu/24.04/cloud --vm incusserver
Launching incusserver
$ incus storage volume create default IncusStorage --type=block size=6GiB
Storage volume IncusStorage created
$ incus storage volume attach default IncusStorage incusserver
$ incus shell incusserver
root@incusserver:~# fdisk -l /dev/sdb
Disk /dev/sdb: 6 GiB, 6442450944 bytes, 12582912 sectors
Disk model: QEMU HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
root@incusserver:~# sudo apt install -y incus zfsutils-linux
...
root@incusserver:~# incus admin init
Would you like to use clustering? (yes/no) [default=no]:
Do you want to configure a new storage pool? (yes/no) [default=yes]:
Name of the new storage pool [default=default]:
Name of the storage backend to use (dir, zfs) [default=zfs]:
Create a new ZFS pool? (yes/no) [default=yes]:
Would you like to use an existing empty block device (e.g. a disk or partition)? (yes/no) [default=no]: yes
Path to the existing block device: /dev/sdb
Would you like to create a new local network bridge? (yes/no) [default=yes]:
What should the new bridge be called? [default=incusbr0]:
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
Would you like the server to be available over the network? (yes/no) [default=no]:
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]:
Would you like a YAML "init" preseed to be printed? (yes/no) [default=no]: yes
config: {}
networks:
- config:
ipv4.address: auto
ipv6.address: auto
description: ""
name: incusbr0
type: ""
project: default
storage_pools:
- config:
source: /dev/sdb
description: ""
name: default
driver: zfs
profiles:
- config: {}
description: ""
devices:
eth0:
name: eth0
network: incusbr0
type: nic
root:
path: /
pool: default
type: disk
name: default
projects: []
cluster: null
root@incusserver:~#
Next we populate the Incus installation with a few alpines. We do this because we want to see these containers again after we recover the storage pool.
root@incusserver:~# incus launch images:alpine/edge alpine1
Launching alpine1
root@incusserver:~# incus launch images:alpine/edge alpine2
Launching alpine2
root@incusserver:~# incus launch images:alpine/edge alpine3
Launching alpine3
root@incusserver:~#
This is where the interesting stuff start. We now want to shutdown the Incus server and remove it. However, the block storage volume will still be there and in good condition, as the server has been shutdown cleanly. Note that the block storage volumes should only be attached to one system at a time.
root@incusserver:~# shutdown -h now
root@incusserver:~# Error: websocket: close 1006 (abnormal closure): unexpected EOF
$ incus storage volume show default IncusStorage
config:
size: 6GiB
description: ""
name: IncusStorage
type: custom
used_by:
- /1.0/instances/incusserver
location: none
content_type: block
project: default
created_at: ...
$ incus delete incusserver
$ incus storage volume show default IncusStorage
config:
size: 6GiB
description: ""
name: IncusStorage
type: custom
used_by: []
location: none
content_type: block
project: default
created_at: ...
$
Next, we launch a new VM that will be used as a new Incus server, then attach back the block storage volume with incus storage volume attach
and install Incus along with the necessary ZFS client utils.
$ incus launch images:ubuntu/24.04/cloud --vm incusserver
Launching incusserver
$ incus storage volume attach default IncusStorage incusserver
$ incus shell incusserver
Error: Instance is not running
$ incus shell incusserver
Error: VM agent isn't currently running
$ incus shell incusserver
Error: VM agent isn't currently running
$ incus shell incusserver
Error: VM agent isn't currently running
$ incus shell incusserver
Error: VM agent isn't currently running
$ incus shell incusserver
Error: VM agent isn't currently running
$ incus shell incusserver
Error: VM agent isn't currently running
$ incus shell incusserver
root@incusserver:~# apt install -y zfsutils-linux incus
...
root@incusserver:~#
Finally, we bring back the old installation data with those three alpines. We run zpool import
, which is a ZFS command that will look for potential ZFS pools and list them by name. The command zpool import default
is the one that does the actual import. The ZFS pool name default was the name that was given by Incus before, when we were initializing Incus. Subsequently, we run incus admin recover
to recover the ZFS pool and reconnect it with this new installation of Incus.
root@incusserver:~# zfs list
no datasets available
root@incusserver:~# zpool list
no pools available
root@incusserver:~# zpool import
pool: default
id: 8311839500301555365
state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:
default ONLINE
sdb ONLINE
root@incusserver:~# zpool import default
root@incusserver:~# zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
default 5.50G 6.80M 5.49G - - 0% 0% 1.00x ONLINE -
root@incusserver:~#
root@incusserver:~# incus admin recover
This server currently has the following storage pools:
Would you like to recover another storage pool? (yes/no) [default=no]: yes
Name of the storage pool: default
Name of the storage backend (zfs, dir): zfs
Source of the storage pool (block device, volume group, dataset, path, ... as applicable): /dev/sdb
Additional storage pool configuration property (KEY=VALUE, empty when done):
Would you like to recover another storage pool? (yes/no) [default=no]:
The recovery process will be scanning the following storage pools:
- NEW: "default" (backend="zfs", source="/dev/sdb")
Would you like to continue with scanning for lost volumes? (yes/no) [default=yes]:
Scanning for unknown volumes...
The following unknown storage pools have been found:
- Storage pool "default" of type "zfs"
The following unknown volumes have been found:
- Container "alpine2" on pool "default" in project "default" (includes 0 snapshots)
- Container "alpine3" on pool "default" in project "default" (includes 0 snapshots)
- Container "alpine1" on pool "default" in project "default" (includes 0 snapshots)
Would you like those to be recovered? (yes/no) [default=no]: yes
Starting recovery...
root@incusserver:~# incus list
+---------+---------+------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+---------+---------+------+------+-----------+-----------+
| alpine1 | STOPPED | | | CONTAINER | 0 |
+---------+---------+------+------+-----------+-----------+
| alpine2 | STOPPED | | | CONTAINER | 0 |
+---------+---------+------+------+-----------+-----------+
| alpine3 | STOPPED | | | CONTAINER | 0 |
+---------+---------+------+------+-----------+-----------+
root@incusserver:~#
Those alpines are in a STOPPED state. Will they start? Sure they will.
root@incusserver:~# incus start alpine1 alpine2 alpine3
root@incusserver:~# incus list -c ns4t
+---------+---------+----------------------+-----------+
| NAME | STATE | IPV4 | TYPE |
+---------+---------+----------------------+-----------+
| alpine1 | RUNNING | 10.36.146.69 (eth0) | CONTAINER |
+---------+---------+----------------------+-----------+
| alpine2 | RUNNING | 10.36.146.101 (eth0) | CONTAINER |
+---------+---------+----------------------+-----------+
| alpine3 | RUNNING | 10.36.146.248 (eth0) | CONTAINER |
+---------+---------+----------------------+-----------+
root@incusserver:~#
In this tutorial we saw how to recover an Incus installation, while the storage pool is intact. We covered the case that the storage pool is ZFS on a block device.
September 10, 2024
OpenUK Awards 2024
Jonathan Riddell
https://openuk.uk/openuk-september-2024-newsletter-1/
https://www.linkedin.com/feed/update/urn:li:activity:7238138962253344769/
Our 5th annual Awards are open for nominations and our 2024 judges are waiting for your nominations! Hannah Foxwell, Jonathan Riddell, and Nicole Tandy will be selecting winners for 12 categories. ?
The OpenUK Awards 2024 are open for nominations until Sunday, September 15.. Our 5th Awards again celebrate the UK’s leadership and global collaboration in open technology!
Nominate now! https://openuk.uk/awards/openuk-awards-2024/
Up to 3 shortlisted nominees will be selected in each category by early October and each nominee will be given one place at the Oscars of Open Source, the black tie Awards Ceremony and Gala Dinner for our 5th Awards held at the House of Lords on 28 November, thanks to the sponsorship of Lord Wei.
September 06, 2024
Windows 3.11 on QEMU 5.2.0
Alan Pope
This is mostly an informational PSA for anyone struggling to get Windows 3.11 working in modern versions of QEMU. Yeah, I know, not exactly a massively viral target audience.
Anyway, short answer, use QEMU 5.2.0 from December 2020 to run Windows 3.11 from November 1993.
An innocent beginning
I made a harmless jokey reply to a toot from Thom at OSNews, lamenting the lack of native Mastodon client for Windows 3.11.
When I saw Thom’s toot, I couldn’t resist, and booted a Windows 3.11 VM that I’d installed six weeks ago, manually from floppy disk images of MSDOS and Windows.
I already had Lotus Organiser installed to post a little bit of nostalgia-farming on threads - it’s what they do over there.
Post by @popeyView on Threads
I thought it might be fun to post a jokey diary entry. I hurriedly made my silly post five minutes after Thom’s toot, expecting not to think about this again.
Incorrect, brain
I shut the VM down, then went to get coffee, chuckling to my smart, smug self about my successful nerdy rapid-response. While the kettle boiled, I started pondering - “Wait, if I really did want to make a Mastodon client for Windows 3.11, how would I do it?”
I pondered and dismissed numerous shortcuts, including, but not limited to:
- Fake it with screenshots doctored in MS Paint
- Run an existing DOS Mastodon Client in a Window
- Use the Windows Telnet client to connect insecurely to my laptop running the Linux command-line Mastodon client, Toot
- Set up a proxy through which I could get to a Mastodon web page
I pondered a different way, in which I’d build a very simple proof of concept native Windows client, and leverage the Mastodon API. I’m not proficient in (m)any programming languages, but felt something like Turbo Pascal was time-appropriate and roughly within my capabilities.
Diversion
My mind settled on Borland Delphi, which I’d never used, but looked similar enough for a silly project to Borland Turbo Pascal 7.0 for DOS, which I had. So I set about installing Borland Delphi 1.0 from fifteen (virtual) floppy disks, onto my Windows 3.11 “Workstation” VM.
Thank you, whoever added the change floppy0
option to the QEMU Monitor. That saved a lot of time, and was reduced down to a loop of this fourteen times:
"Please insert disk 2"
CTRL+ALT+2
(qemu) change floppy 0 Disk02.img
CTRL+ALT+1
[ENTER]
During my research for this blog, I found a delightful, nearly decade-old video of David Intersimone (“David I”) running Borland Delphi 1 on Windows 3.11. David makes it all look so easy. Watch this to get a moving-pictures-with-sound idea of what I was looking at in my VM.
Once Delphi was installed, I started pondering the network design. But that thought wasn’t resident in my head for long, because it was immediately replaced with the reason why I didn’t use that Windows 3.11 VM much beyond the original base install.
The networking stack doesn’t work. Or at least, it didn’t.
That could be a problem.
Retro spelunking
I originally installed the VM by following this guide, which is notable as having additional flourishes like mouse, sound, and SVGA support, as well as TCP/IP networking. Unfortunately I couldn’t initially get the network stack working as Windows 3.11 would hang on a black screen after the familiar OS splash image.
Looking back to my silly joke, those 16-bit Windows-based Mastodon dreams quickly turned to dust when I realised I wouldn’t get far without an IP address in the VM.
Hopes raised
After some digging in the depths of retro forums, I stumbled on a four year-old repo maintained by Jaap Joris Vens.
Here’s a fully configured Windows 3.11 machine with a working internet connection and a load of software, games, and of course Microsoft BOB 🤓
Jaap Joris published this ready-to-go Windows 3.11 hard disk image for QEMU, chock full of games, utilities, and drivers. I thought that perhaps their image was configured differently, and thus worked.
However, after downloading it, I got the same “black screen after splash” as with my image. Other retro enthusiasts had the same issue, and reported the details on this issue, about a year ago.
does not work, black screen.
It works for me and many others. Have you followed the instructions? At which point do you see the black screen?
The key to finding the solution was a comment from Jaap Joris pointing out that the disk image “hasn’t changed since it was first committed 3 years ago”, implying it must have worked back then, but doesn’t now.
Joy of Open Source
I figured that if the original uploader had at least some success when the image was created and uploaded, it is indeed likely QEMU or some other component it uses may have (been) broken in the meantime.
So I went rummaging in the source archives, looking for the most recent release of QEMU, immediately prior to the upload. QEMU 5.2.0 looked like a good candidate, dated 8th December 2020, a solid month before 18th January 2021 when the hda.img
file was uploaded.
If you build it, they will run
It didn’t take long to compile QEMU 5.2.0 on my ThinkPad Z13 running Ubuntu 24.04.1. It went something like this. I presumed that getting the build dependencies for whatever is the current QEMU version, in the Ubuntu repo today, will get me most of the requirements.
$ sudo apt-get build-dep qemu
$ mkdir qemu
$ cd qemu
$ wget https://download.qemu.org/qemu-5.2.0.tar.xz
$ tar xvf qemu-5.2.0.tar.xz
$ cd qemu-5.2.0
$ ./configure
$ make -j$(nproc)
That was pretty much it. The build ran for a while, and out popped binaries and the other stuff you need to emulate an old OS. I copied the bits required directly to where I already had put Jaap Joris’ hda.img
and start
script.
$ cd build
$ cp qemu-system-i386 efi-rtl8139.rom efi-e1000.rom efi-ne2k_pci.rom kvmvapic.bin vgabios-cirrus.bin vgabios-stdvga.bin vgabios-vmware.bin bios-256k.bin ~/VMs/windows-3.1/
I then tweaked the start
script to launch the local home-compiled qemu-system-i386
binary, rather than the one in the path, supplied by the distro:
$ cat start
#!/bin/bash
./qemu-system-i386 -nic user,ipv6=off,model=ne2k_pci -drive format=raw,file=hda.img -vga cirrus -device sb16 -display gtk,zoom-to-fit=on
This worked a treat. You can probably make out in the screenshot below, that I’m using Internet Explorer 5 to visit the GitHub issue which kinda renders when proxied via FrogFind by Action Retro.
Share…
I briefly toyed with the idea of building a deb of this version of QEMU for a few modern Ubuntu releases, and throwing that in a Launchpad PPA then realised I’d need to make sure the name doesn’t collide with the packaged QEMU in Ubuntu.
I honestly couldn’t be bothered to go through the pain of effectively renaming (forking) QEMU to something like OLDQEMU so as not to damage existing installs. I’m sure someone could do it if they tried, but I suspect it’s quite a search and replace, or move the binaries somewhere under /opt
. Too much effort for my brain.
I then started building a snap of qemu as oldqemu - which wouldn’t require any “real” forking or renaming. The snap could be called oldqemu
but still contain qemu-system-i386
which wouldn’t clash with any existing binaries of the same name as they’d be self-contained inside the compressed snap, and would be launched as oldqemu.qemu-system-i386
.
That would make for one package to maintain rather than one per release of Ubuntu. (Which is, as I am sure everyone is aware, one of the primary advantages of making snaps instead of debs in the first place.)
Anyway, I got stuck with another technical challenge in the time I allowed myself to make the oldqemu snap. I might re-visit it, especially as I could leverage the Launchpad Build farm to make multiple architecture builds for me to share.
…or not
In the meantime, the instructions are above, and also (roughly) in the comment I left on the issue, which has kindly been re-opened.
Now, about that Windows 3.11 Mastodon client…