New Funded Project: Capsicum Improvements

The FreeBSD Foundation is pleased to announce that Pawel Jakub Dawidek has been awarded a grant to develop a comprehensive userspace framework for writing Capsicum-based applications, building on the kernel features originally developed by the University of Cambridge and Google Research.

This framework will include a Capsicum runtime linker and component library providing sandboxed versions of key higher-level system libraries. Components will both be sandboxed, improving resistance to vulnerabilities, and also easily available for delegation to sandboxed applications, such as the Chromium web browser. The prototype libcapsicum developed by Cambridge will be analyzed and updated based on lessons learned in implementing Capsicumised software packages, such as hastd and auditdistd. Funding for this project will be provided by the FreeBSD Foundation matched 100% by the Google Open Source Program Office, in support of open source technology transition of Capsicum.

"A continuing challenge in security is to find solutions that not only fix the problems but also can be applied to existing technologies: attractive though the notion is, we are not going to persuade the world to rewrite everything! This is why we at Google are pleased and excited to support the continuing development of Capsicum, which radically improves the security of UNIX based systems whilst allowing a continuous migration path from today's mechanisms to tomorrow's," said Ben Laurie, Google Senior Staff Software Engineer.

"I'm very excited to be able to work on Capsicum. Some of my software is already using Capsicum, so I'm fully aware of the great potential of this framework. This technology is so much superior than the current attempts to provide sandboxing using tools like chroot(2) or unprivileged user credentials. No matter how corny it sounds, I strongly believe Capsicum can make the Internet a safer place." said Pawel.

This project will conclude in August, 2012

BSDCan Trip Report: Florian Smeets

The next trip report is from Florian Smeets:

I arrived in Ottawa on Tuesday afternoon, together with Giovanni Trematerra who was on the same flight as I. After dropping our luggage off at the Residency one of the first people we met was Beat Gaetzi, my gecko@ companion. After a very short discussion it was decided to go to the Royal Oak. Over the course of the evening lots of developers showed up there. It was good to see some of those again whom I had already met at EuroBSDCon and FOSDEM and it was great to finally meet people I only knew from email/IRC.

Early the next day the first day of the devsummit started. I attended the ports session which went on for almost all day. We talked about a lot of things, spanning from what was done in the last year to new features which we will see in the future. In the evening I attended the vendor summit which was quite interesting. A lot of FreeBSD using companies discussed what they are working on, which features they could share with the community, and what they need. I had to leave the vendor summit a few minutes early as Thomas Abthorpe had scheduled a dinner meeting, to take a few of the ports committers out to try a Canadian dish called Poutine.

The second day started with the Admins session. One of the topics was pkgng package distribution which took quite some time. In the Toolchain session there was a lot of talk about our switch to LLVM/clang. The next session was the working group summaries, where all the session leaders presented the accomplishments of the different groups of the previous 2 days. The last session was brainstorming for the 10.0 release, which was one of the most interesting sessions for me. To see all these features and improvements that should/could be in 10.0.

On day 3 the conference started. After a funny opening session which was started by a bagpiper it went to "An Overview of Locking in the FreeBSD kernel", where Kirk McKusick gave a high level overview of the different locking primitives one can use, and when/how to use them. After that I headed over to the FreeBSD devsummit track and stayed there for the remainder of the day. The topics included Documentation, Kernel debugging, CAM Target layer, an update on 802.11 in FreeBSD, Capsicum, FusionIO and work on callout(9).

Later that evening I sat down with Alan Cox and Attilio Rao and listened to both of them planning their next work on our VM system. This was a great experience. I learned a lot, though i was not able understand all of it.

Inspired by Julian Elischer's talk on FusionIO and FreeBSD (which reminded me of the fact that we have a machine in the FreeBSD cluster with one of those cards), I sat down with Julian to get the card going. I had talked to Beat the day before about the ports migration to svn and he told me that it took almost 2 days to import the converted repository into svn. After some experimenting the result was that it takes only 3 hours on a FusionIO card. We will use this for doing the real migration to keep the outage as short as possible.

The last day I attended Optimizing ZFS for Block Storage, went to the OpenBSD network stack evolution talk to see what they are up to, and as a last talk I went to see the pkgng talk to see if there were any new features I did not know about yet.

It was a great trip. I learned a lot, it was great to put many faces to names (too many to list them here).

I want to sincerely thank the FreeBSD Foundation for enabling me to go to the devsummit and BSDCan 2012.

BSDCan Trip Report: Ben Haga

The next trip report is from Ben Haga:

Without the generous support of the FreeBSD Foundation my trip to BSDCan 2012 would not have been possible. It was a great experience and I cannot thank the Foundation enough for the opportunity. Attending BSDCan was an excellent chance to meet the folks currently driving the future of FreeBSD. The experience of working alongside each of these esteemed developers is like getting to know, as Dan Langille put it, an "extended family."

I arrived in Ottawa on the evening of May 8th. After checking in at the Residences, I went, along with several Cluster Administrators, to an informal meeting at the Royal Oak. As I learned, drink and food at the Royal Oak is a fun BSDCan tradition.

On the first day of the Dev Summit, I met my fellow cohorts in the FreeBSD.org Cluster Administration group. This was a great experience and opportunity to get to know the folks I work with on a regular basis better. Throughout the day and stay in Ottawa, we talked over current issues and items to handle moving forward with cluster-related work and activities.  

May 9

The Dev Summit started off with a bang. First, was the Developer Summit registration, followed by an introductory session by John Baldwin. Shortly following the introductory session, I met up with fellow cluster administrators in order to spend time with one another and get to know each other better. In addition to getting to know each other face-to-face, we discussed the current state of the cluster(s).

After a successful hacking session with my colleagues, I headed to the PC-BSD-oriented Desktop session with Kris Moore and Dru Lavigne. This proved to be a valuable session for those running or interested in running PC-BSD on their desktop. Not only were we able to interact with those heavily involved in the PC-BSD project, but we were able to provide valuable feedback as well. Following the day's activities, a number of folks from the Ports crew as well as the Cluster Administrators group headed to the Clocktower Market to hang out and discuss happenings in the FreeBSD universe.  

May 10

The second day of the Developer Summit was no less exciting than the first. In the morning, after heading to breakfast at the Second Cup, I headed to the Admins working group. Although the meeting drew heavy attendance and was at times difficult to work as an entire group, we were successful in hammering out a number of details for future plans. During this time, the Yahoo! contingent was able to ascertain details for future deployment in the new home for the FreeBSD.org cluster hosted at Yahoo!. This will result in greater capacity and more robust infrastructure for the road ahead.

Later in the day, I spoke with several developers and through discussion found an opportunity to contribute to the NTP tree, which further piqued my interest in updating the NTP software in base. With this responsibility and past involvement with the software, I will be able to bring the NTP software in base to a modern state with bug fixes and improvements. At the end of the day, it was time for early BSDCan 2012 conference registration and a few beverages at The Royal Oak. Later in the evening, I met up with a party of others on the 15th floor hacker's lounge for general discussion and otherwise good time!  

May 11, 12

After the long evenings, nights, and evenings spent hacking on various projects and working with other FreeBSD enthusiasts on a number of topics, it was time for the BSDCan 2012 conference to begin. I attended the morning's introductory speech given by Dan Langille. From there, I hustled to the much anticipated talk by Kirk McKusick entitled, "An Overview of Locking in the FreeBSD Kernel." After Kirk's talk ended, I made it to Pawel Jakub Dawidek's talk, "auditdistd - Secure and reliable distribution of audit trail files", Ivan Voras' "Bullet Cache Balancing speed and usability in a cache server", and Russell Cattelan's "Fast reboots with kload."

In the evening, I attended Julian Elischer's talk, "FusionIO and FreeBSD", followed by "Work on callout(9)" by Davide Italiano. In addition, I attended "pkgng" by Baptiste Daroussin and "Solaris Boot Environments for FreeBSD" by Nikolai Lifanov. Overall, the talks were engaging and I found myself hoping for more time to capture more of the fantastic talks and presenters. Of course, no trip to BSDCan nor the Dev Summit would be complete without late night hacking and tunes in the first and fifteenth floor hacker lounges at the Residences. These were extremely productive and fun as well.  

May 13

After a long night of hacking with a number of esteemed compatriots, it was time for my journey in Ottawa to come to an end. BSDCan 2012 was a great and productive experience and I look forward to coming back to future conferences!

BSDCan Trip Report: Davide Italiano

The next trip report is from Davide Italiano:

A couple of week ago, thanks to the grant I received from the FreeBSD Foundation, I was able to attend the FreeBSD Devsummit and the BSDCan conference in Ottawa. This was my first BSD-related event and I was very excited. I'm pretty satisfied with the whole experience, that I'll try to explain in the next paragraphs.

I arrived in Ottawa on 7th May, and I left the city on 13th May. I stayed in the University of Ottawa dormitory, sharing a room together with Ollivier Robert. The trip was really nice for a fair amount of reasons. For example, it gave me the possibility to talk/work/deal with some people I've only interacted with on IRC or FreeBSD mailing lists. Then, considering that I've never been in Canada, it was a nice occasion to test my English knowledge skills (and they weren't as bad as I expected) and to interact with people from different cultures.

The DevSummit itself was really intense. Lots of interesting ideas have been presented during the working groups. On the morning of the 9th, I attended the Offload Engines workgroup coordinated by Jim Harris. He (with others) explained the motivations for an offload framework in FreeBSD and what the issues in the implementation of it. I was suprised because the people talking were so good that I ended up with understanding everything even though I wasn't really aware of the problem before attending. I hope Jim and his team at Intel will continue this work. If you're interested, the slides (as well as the material of all the working groups and the talks) may be found on  the DevSummit Page of the FreeBSD Wiki.

On the afternoon of the 9th I attended the Desktop Workgroup and I talked with some of the PC-BSD guys (in particular Kris Moore), proposing to them some features I'd like to see included in the next PC-BSD release (among others, better wireless support and automated kernel crash report). They were really happy to have feedback on their work.

During the evening the Vendor summit took place, moderated by gnn@. Consumers requested what they need from FreeBSD, and developers tried to get companies to bring code back to the community. I asked for the Ivy Bridge hwpmc support which is still missing and needs specific hardware to work on. It seems that we'll get access to some Ivy bridge machines soon, so it was worthwhile to ask.

At the Virtualization workgroup (which was on the morning of the 10th) some guys from NetApp presented the status of BHyVe. An overview of the shape of Xen in FreeBSD was also discussed. In the afternoon (of 10th) the working group reports were presented and I was able to know what was discussed in the working groups I didn't attend due to overlaps. As a university student, the report from Robert Watson on "how to teach an OS course using FreeBSD" caught my attention. The methodologies explained by Robert are pretty different from the ones used in Italy (at least in the university I attended), and as always, looking at things from another point of view helps.

The BSDCan conference had lots of useful talks: in particular I liked the one from Kirk McKusick on FreeBSD Locking and the one on Tuning ZFS by Justin Gibbs and Will Andrew. During the Devsummit track of the BSDCan conference I presented my work for the Google Summer of Code 2012 and I got nice feedback from lots of people (in particular, from my mentor at GSoC, Alexander Motin, which was really helpful and helped me in the design of the project I'm currently working on). A video of the presentation may be found on youtube at this link. I was a bit nervous because it was my first talk, but I hope you'll enjoy it in any case.

In the evening, every day, developers shared food and beer, and later on, hacking sessions were set up in some room at the Uottawa dormitory.

To sum up, this experience was fantastic in any sense. I hope to attend other conferences in the future. Thanks again to the FreeBSD foundation for the grant and all the FreeBSD Developers which shared this adventure with me.

Foundation at SouthEast LinuxFest

There will be a BSD booth in the Expo area at SouthEast LinuxFest this Saturday, June 9 in Charlotte, NC. This event is free to attend so if you are in the area drop by to say hi, discuss the Foundation's funded projects, check out the cool swag, or make a donation to the Foundation.

BSDCan Trip Report: Warren Block

The next trip report is from Warren Block:

I arrived Tuesday night and met Glen Barber and Benedict Reuschling, my FreeBSD mentors, for the first time in person.  Many well-known FreeBSD Big Names were there already.

On Wednesday morning, the FreeBSD developer summit began.  We split into working groups, and I attended the documentation group.  Hiroki Sato talked about the welcome conversion of the doc repository from CVS to SVN.  We talked about work on a new print version of the Handbook and about ways to encourage translations of the documentation to some of the more popular languages that are missing, particularly Spanish, Italian, and Chinese.  We also talked about making documentation versioned, with different books or possibly just different sections for, say, FreeBSD 8 and 9.  Bludgeoning the projector into submission allowed me to give a live demonstration of igor (textproc/igor), a documentation proofreading tool that aims to make writing easier for documentation authors.  The three-hour session went by quickly.

Wednesday afternoon was a tutorial on PF by Peter Hansteen.  My suspicion was that I was not using it to full potential, and it turns out I was right.

Thursday morning, Eric Crist and Thomas Johnson gave a tutorial on OpenVPN.  This was something I'd been curious about, and the presentation showed that using OpenVPN really was surprisingly simple.

Thursday afternoon, the working groups presented summaries.  Much was accomplished, and time limits frequently cut off the reports.

BSDCan talks began on Friday.  There were four separate tracks with compelling talks in each, and choosing was somewhat difficult. Fortunately, the talks were recorded, both audio and the video output from the projector, and are being put on Youtube.

My half-hour presentation, called "Automated Documentation Proofreading", was about the need, design goals, and use of igor.  In spite of the title, a full crowd showed up, and the presentation went well. Afterwards, Rob Gigliotti suggested using antispam software for analysis of writing, and Jeremy Reed suggested the use of traditional Unix tools like style(1) and diction(1).

Some of the crowd at my presentation may have just been waiting for Bjoern Zeeb's "Kernel Debugging Tricks" talk, which followed mine in the same room.  It was standing room only, and the debugging advice applicable to more than just kernel debugging.

Later that day, Andrew Pantyukhin's "FreeBSD Unified Deployment and Configuration Management" was really thought-provoking.  He has a smart and lightweight approach, worth considering even if you only have a few computers rather than hundreds.

Saturday began with Tom Judge's "Building a FreeBSD based Virtual Appliance", using a combination of methods and technologies that could be really useful to all sorts of FreeBSD users.  Also touched on was the FreeBSD Admin project, the web-based admin code from FreeNAS split out into a separate, reusable component.

Saturday afternoon, Baptiste Daroussin presented a standout talk describing the new pkgng package management system.  I'd heard of it but not tried it.  There were demonstrations of equivalents with the current package system, showing that pkgng really will not have much of a learning curve.  When he showed how packages in a jail could be upgraded easily from outside the jail, the audience broke into applause.  This happened several more times, followed by thunderous applause at the end.

Sunday, another conference was held with documentation groups from FreeBSD, the PostgreSQL project, and other interested parties.  Although there were a few problems with rooms and the availability of network connections, there were constructive discussions on translation software, difficulties checking out the doc tree (to be made easier with svn), encouraging submission of doc bug reports (possibly by "gamifying" them with points or a karma system), keeping submitters involved so bug reports don't seem to go into a black hole, and possibly having a "doc lounge" where documentation committers would help fix doc bugs at FreeBSD events as you wait.  Interaction with other projects can be very symbiotic.  We should do it more.

Throughout the dev summit and BSDCan, a "hacker lounge" was available where people could get together and just work on things, and they did, often skipping sleep.

Meeting so many FreeBSD dignitaries was amazing.  As Ivan Voras (who I also met) says on his blog, "It is very close to an ideal meritocracy."

My thanks to the FreeBSD Foundation for their assistance with a travel grant, and to Benedict Reuschling and Glen Barber for patiently enduring my endless questions.

BSDCan Trip Report: Thomas Abthorpe

The next trip report is from Thomas Abthorpe:

Third time is the charm. That is what is said when you have failed two times before and finally got it right on the third try. To put a more postive spin on it, the third time is the best one yet! I have been fortunate and privileged to travel to Ottawa for BSDCan three times now, courtesy of the FreeBSD Foundation. My first trip was in 2009. I was the new kid on the block, and felt excited and overwhelmed being surrounded by FreeBSD people. My second trip was in 2011, by this time I had been established as portmgr-secretary@, and felt quite comfortable in these now familiar surroundings and had an established niche. Here in 2012, on my third trip, I feel I am now beyond my sophmore period in the FreeBSD Project, and most importanly, able to attend the event through it's entirety, something I had not been able to do in years gone by.

I had four full days to spend in Ottawa, and I was determined to make the most of it. I arrived on campus near midnight on Tuesday evening and was greated by some familiar and not so familiar faces outside of the residence. Baptiste Daroussin and Beat Gaetzi, my fellow Port Managers, were there among the crowd of well wishers. I met Baptiste last year, and was meeting Beat for the very first time, and lucky me, Beat is almost as big a hockey fan as I am! The three of us took the next hour to socialise and get (re)acquainted.

Wednesday was the start of the Developers Summit. After the introductory session was held, developers were sent off to their respective working groups. Three Ports working groups were scheduled commencing after 11:00, so the Ports Management Team opted to take advantage of having an unprecedented five members (Mark Linimon, Beat Gaetzi, Baptiste Daroussin, Erwin Lansing and myself) present to meet during the free time. We managed to link in Florent Thoumie via Skpe, and discussed matters relating to the future of the Ports infrastructure. For the remainder of the Ports sessions, Mark Linimon facilitated the discussions. Other members of the team took turns to talk about their specific areas of interest. That night for supper, I took a group of European ports committers (Baptiste Daroussin, Julien Lafayye, myself, and Florian Smeets) out to indulge in Canadian comfort food, commonly called poutine. They have heard me talk about it in IRC for a long time, and decided it was finally time to give it a try. They said they came to Canada for BSDCan, but decided to stay for the poutine.

Day two of the Developers Summit was a little more subdued, instead of being focused intently on working groups, this day was more informational updates from various presenters. I attended the Admins session to hear what our clusteradm@ folk were up to, plus get a situation report around “Project Evil”. Following that, I attended the Toolchain update, so I could get a sense of when Clang would get turned on by default, and the possible spin off repercussions on the Ports infrastructure. Following the lunch break, the Working groups reports were presented. In a perfect world, I would have attended all the sessions, but alas, I could not, so I got a summary of what everybody was up to. I was particularly encouraged and inspired by the presentation that Benedict Reuschling, bcr@, gave from the Documentation working group. They have identified a need for new documentation committers, and a desire to try new methods to encourage and mentor new committers into the Doc tree. For my third conference I was able to attend a dot zero planning session. In many of our professional lives, we have little or no control nor influence over what goes into a .0 release, and here I am listening in on what would be done. This is still an amazing experience for me, and I just sit there like a little child in the presence of his hero, taking in the whole experience.

Day one of the conference saw a whole new opening, a bagpiper leading a kilted Dan Langille into the the lecture theatre. This was just the energy injection the crowd needed to kick start
the day! Keeping with my ports theme, I attended Mark Linimon's s Progress in FreeBSD Ports session, followed by Steve Wills' Ports testing session. My third session of the day was something I can hope to bring back to my employer, FreeBSD on MS Hyper-v. This is one of the few times in which my day job co-incides. Following the lunch break, I took in Benedict Reushling's talk about on the Google Code-in. I find it extremely encouraging to hear about teenaged students participating in open source. I was especially pleased to see, as a result of the project, that we benefitted from getting a new documentation committer. The final session I attended was DNSSEC. I have been doing some light reading on this in the last year, and dabbling with the notion of trying to set it up. My biggest take away from this session was to be extra special careful, as you can really do harm to yourself if you do not know what you are doing.

Day two of the conference, it was already Saturday, and I was really finding that the wear and tear of the week was catching up to me! I attended Tom Judge's session on Building a FreeBSD based Virtual appliance. His format was tutorial like in it's approach, and I found the step by step instructions particularly useful! I hope to be able to use this approach to prototype some envrionments in the future. The final talk I attended was Baptiste Daroussin's presentation of pkgng, the Next Generation of ports management. His session last year during the Developer's Summit was the proof of concept, this year he had a working demonstration that (hopefully) won over a whole new group of converts!

The Developers Summit and the Conference is the tangibles that I can bring forward as part of my contribution to the FreeBSD project. The intangibles is; the time spent in the Hacker's Lounge and meeting somebody new; or going to breakfast with a group of people you have never met before; or introducing people to the joys of poutine. This was by far the best BSDCan I have ever attended, and I am very grateful to the FreeBSD Foundation for sponsoring me to attend.

BSDCan Trip Report: Brooks Davis

The next trip report is from Brooks Davis:

I arrived in Ottawa on the 7th and spent the 7th and 8th on tourist activities and meeting informally with fellow developers. On the 8th the majority of Dev Summit attendees met at the Royal Oak pub for dinner.

May 9th

The Dev Summit kicked off with registration followed by an intro session by John Baldwin.  I spent some time talking with other developers before attending a research group conference call.  After lunch I attended the network stack session.  We covered quite a bit of ground with major focuses on the need for an improved design for mbufs and on splitting the layer 2 and layer 3 portions of the network stack more cleanly to better manage the overhead at each level and to better align our data structures with the ways FreeBSD is used.  The session was extremely popular with at least half of the developers at the summit attending.  A larger room or more strict attendance control might have been a good thing for the room's climate control.

After a full day of developers summit, a large portion of us stayed on for the vendor summit.  At the vendor summit we produced the now usual lists of things different organizations have contributed, would like to contribute, and needed.  The summit felt productive and upbeat with lots of organizations having things to share and a strong interest in joint projects where practical.

May 10th

On the second day of the Dev Summit I started out in the Admins meeting. It suffered from excessive attendance, probably due to being in the primary room, but was reasonably productive. It was followed by the Toolchain summit which was very well attended. I ran that meeting largely as an open discussion of status and plans. The main short term toolchain issue is that we need a plan and timeline to throw the switch for clang to be the default compiler in 10 for at least i386 and amd64. That depends on more ports being fixed and/or a better framework for a switchable ports compiler. Work is in progress on both, but we're probably approaching the point where we need to set a schedule for the switch to drive the
final cleanups.

The toolchain summit was followed by lunch, working group reports, and a discussion of 10.0-RELEASE.  We then had an excellent communal Thai food dinner.

May 11th and 12th

On the first day of the main conference I attended the talks "An Overview of Locking in the FreeBSD Kernel" by Kirk McKusick, "auditdistd-- Secure reliable distribution of audit trail files" by Pawal Jakub Dawidek, "State of 802.11 in FreeBSD" by Adrian Chadd, "Capsicum" by Robert Watson, "FusionIO and FreeBSD" by Julian Elischer, and "Work on callout(9)" by Davide Italiano. On the second day I attended "Overview of Amazon Web Services" by Randi Harper where she announced an AWS based portsnap mirror, "pkgng - Modernising FreeBSD package management" by Baptiste Daroussin, and the closing session. I would have liked to attend several other talks including "Ethernet Switch Framework" by Stefan Bethke and "Solaris Boot Environments for FreeBSD" by Nikolai Lifanov but several conflicts arose. All in all the talk schedule was engaging and I several times I wished for more copies of me to attend the talks.

BSDCan 2012 was another great experience.  I hope that next year's 10th edition can live up to the past nine years!