-
 KDE-Apps.org Applications for the KDE-Desktop 
 GTK-Apps.org Applications using the GTK Toolkit 
 GnomeFiles.org Applications for GNOME 
 MeeGo-Central.org Applications for MeeGo 
 CLI-Apps.org Command Line Applications 
 Qt-Apps.org Free Qt Applications 
 Qt-Prop.org Proprietary Qt Applications 
 Maemo-Apps.org Applications for the Maemo Plattform 
 Java-Apps.org Free Java Applications 
 eyeOS-Apps.org Free eyeOS Applications 
 Wine-Apps.org Wine Applications 
 Server-Apps.org Server Applications 
 apps.ownCloud.com ownCloud Applications 
--
-
 KDE-Look.org Artwork for the KDE-Desktop 
 GNOME-Look.org Artwork for the GNOME-Desktop 
 Xfce-Look.org Artwork for the Xfce-Desktop 
 Box-Look.org Artwork for your Windowmanager 
 E17-Stuff.org Artwork for Enlightenment 
 Beryl-Themes.org Artwork for the Beryl Windowmanager 
 Compiz-Themes.org Artwork for the Compiz Windowmanager 
 EDE-Look.org Themes for your EDE Desktop 
--
-
 Debian-Art.org Stuff for Debian 
 Gentoo-Art.org Artwork for Gentoo Linux 
 SUSE-Art.org Artwork for openSUSE 
 Ubuntu-Art.org Artwork for Ubuntu 
 Kubuntu-Art.org Artwork for Kubuntu 
 LinuxMint-Art.org Artwork for Linux Mint 
 Arch-Stuff.org Art And Stuff for Arch Linux 
 Frugalware-Art.org Themes for Frugalware 
 Fedora-Art.org Artwork for Fedora Linux 
 Mandriva-Art.org Artwork for Mandriva Linux 
--
-
 KDE-Files.org Files for KDE Applications 
 OpenTemplate.org Documents for OpenOffice.org
 GIMPStuff.org Files for GIMP
 InkscapeStuff.org Files for Inkscape
 ScribusStuff.org Files for Scribus
 BlenderStuff.org Textures and Objects for Blender
 VLC-Addons.org Themes and Extensions for VLC
--
-
 KDE-Help.org Support for your KDE Desktop 
 GNOME-Help.org Support for your GNOME Desktop 
 Xfce-Help.org Support for your Xfce Desktop 
--
openDesktop.orgopenDesktop.org:   Applications   Artwork   Linux Distributions   Documents    LinuxDaily.com    Linux42.org    OpenSkillz.com   
 
Artwork
News
Groups
Knowledge
Events
Forum
People
Jobs
Register
Login

-
- News . 
0
votes
click to vote up

Jono Bacon: Ubuntu In a Nutshell: App Upload Process


Published Aug 21 2013 via RSS

This article is part of a series of blog posts covering the many different areas of work going on in Ubuntu right now. See the introduction post here that links to all the articles.

In this article I am going to discuss some improvements we making to significantly simply and speed up how app devs get their apps into Ubuntu.

You can think of an Ubuntu system as two layers of software:

  1. The System – the core Operating System includes the system software itself that is required to boot the device, bring up services (e.g. networking/bluetooth/sound), and display the user interface.
  2. The Applications – these are the applications that run on top of the system where you spend most of your time as a user.

When we started working on Ubuntu back in 2004 the system and the applications where closely intermingled. We basically synced the Debian archive with Ubuntu, applied a bunch of patches, and whatever was in the Debian and Ubuntu archives was available as applications.

There were some inherent problems with this approach:

  • To get an app into Ubuntu you need to create a .deb file (a Debian package). Debian packages are really designed to be used by people who build Operating Systems, and not regular app developers.
  • This approach meant that you could write an app with any software toolkit/platform that was in the archive. Unfortunately this results in a really inconsistent end-user experience – we have tried our best to support GTK and Qt apps, but these apps look and work quite differently, and it spreads the development team thinl trying to cater to all tastes.
  • If an app developer wants to get an app into Ubuntu they have to either (a) be an Ubuntu developer with upload access or (b) convince a developer to get their app in. This doesn’t scale.

With these issues we tried to remedy the problem by creating the Application Review Board; a community group who would review packages with a simplified criteria. Unfortunately, despite the best efforts of the ARB, they were never really set up for success due to the technical limitations in the platform.

The primary technical limitation the ARB faced is that the Ubuntu system has traditionally lacked sand-boxing to protect it from naughty applications doing something…well, naughty. This meant that every app they reviewed would need a full code review for every version submitted. So, an app developer could release 1.0 and then release 1.1 a few weeks later and this would require two full code reviews. Not only this, but Debian packages have something called maintainer scripts that can theoretically do anything as root, so the packaging needed checking too.

For a group of volunteers with a growing list of submitted apps the ARB quickly fell behind. This resulted in frustration all around – for the ARB, for app devs, and for those of us working to refine and improve Ubuntu.

The New App Upload Process

So, the previous system had the following primary limitations:

  • Packaging was complicated.
  • No app sandboxing, so apps needed a full code review.
  • Maintainer scripts can do naughty things, so they needed a full review.
  • Apps took a long time to be reviewed due to the technical limitations above.

We started building a new specification for the new process some time ago. My team wrote the initial spec with input from a range of teams, and this has formed the foundation for the work that is reaching maturity now. Let’s cover how it works.

There are three primary pieces to this work, click packages, sand-boxing, and uploading.

Click Packages

Firstly, remember how earlier I mentioned two layers in an Ubuntu system:

  1. The System – the core Operating System includes the system software itself that is required to boot the device, bring up services (e.g. networking/bluetooth/sound), and display the user interface.
  2. The Applications – these are the applications that run on top of the system where you spend most of your time as a user.

The Debian packaging format provides a remarkable range of features that are really designed for the former, for building Operating Systems. For the latter, app devs who simply want to package their code into something that can be loaded by users, we have created a new, simplified format for packages. It is called the click package format.

Click packages are simple to create: when you have your project open in the Ubuntu SDK you can press a button to generate a click package. Done! Click packages also don’t include maintainer scripts: they are simply not needed for the vast majority of apps, so, that already removes a security risk and something that needs reviewing.

A key benefit of click packages also is that they don’t include full dependency resolution. Many of you will be familiar with running apt-get update on your system. Well, this syncs the list of packages from the archive and figures out all the dependencies and how they map out (e.g. knowing that GIMP requires GTK, which in turn requires X). This takes quite some time and doesn’t scale to thousands of packages that getting updated all the time.

With a click package the software simply depends on the Ubuntu SDK. This means we don’t need to worry about all that complex dependency resolution: we know the dependency, the Ubuntu SDK. Additional dependencies can simply be bundled into the package. Also, instead of maintaining a list of packages on the system…they are on a web service. You need a connection to download the package anyway, so why not ask a service which packages are available?

Sand-boxing

With a simple and efficient package format all set we next have the issue of sand-boxing.

This is more work than you might imagine. The team identified a big list of sand-boxing requirements that were needed for us to be sure an app could be run without a code review. This is not only protecting the system from inappropriate calls, but also handling issues with other components (e.g. sniffing keyboard events in X apps).

Well, the awesome Ubuntu Security Team has been working on full sand-boxing throughout the system and most of this work has been completed. This has also influenced some other design decisions: as an example, Mir (our new display server) is designed to not expose the keyboard event sniffing issue X has faced.

The basic result of this work is that the security team have an app called evil app that does a series of naughty things, and the sand-boxing protects the system from such naughtyness.

With this sand-boxing in place it essentially mitigates the need for a full code review, which was the primary bottleneck previously. This combined with click packages not having maintainer scripts and complex dependency chains makes reviews much easier and more efficient.

Uploading

The process for uploading an app to Ubuntu is going to remain largely the same as it works generally pretty well. As before you can select a package to upload, set information about the app, add screenshots and more and then it will hit a review queue before it appears to Ubuntu users.

The good news is that because the review only really require a check of permissions, meta-data, and a few other simple bits (thanks to click packages and sand-boxing), the review can be done in under 15 minutes as opposed to multi-day code reviews. This means apps should go in more quickly.

Once your app is in the archive you will be able to search for it across all Ubuntu devices. Currently all apps will be displayed, but we are adding features to only show apps that will work on that device (e.g. only show apps with a phone UI on the phone).

Are We There Yet?

So where do we stand with all this goodness? Fortunately, we are getting close.

The click package format is largely complete and we have tools for creating click packages in the SDK and installing them on a system.

Much of the sand-boxing work has been largely completed, although one caveat is that because we are working on Mir as our new display server, we are not investing in fixing keyboard sniffing in X (which would be a huge amount of work). This means that we won’t release this system on the desktop until Mir is on by default (around 14.10). Our existing system will be in place until then.

The app upload piece is online and largely complete (although going through a series of reviews), and the scope for browsing new click packages is on the mobile images although isn’t quite yet hooked up.

You can see an early video demo of this working below:

Can’t see it? See it here!

Our goal is to have this full system in place for mobile in 13.10, and for desktop in 14.10. This will all make getting apps into Ubuntu quicker, easier, and more rewarding than ever before.



BackRead original postSend to a friend

Add comment

Add comment
Show all posts




-
 
 
 Who we are
Contact
More about us
Frequently Asked Questions
Register
Twitter
Blog
Explore
Artwork
Jobs
Knowledge
Events
People
Updates on identi.ca
Updates on Twitter
Facebook App
Content RSS   
Events RSS   

Participate
Groups
Forum
Add Artwork
Public API
About GNOME-Look.org
Legal Notice
Spreadshirt Shop
CafePress Shop
Advertising
Sponsor us
Report Abuse
 

Copyright 2003-2014 GNOME-Look.org Team  
All rights reserved. GNOME-Look.org is not liable for any content or goods on this site.
All contributors are responsible for the lawfulness of their uploads.
GNOME and the foot logo are trademarks of the GNOME Foundation.