The December issue of the FreeBSD Foundation Newsletter has been published:
FreeBSD has a flourishing community. Innovative technology is packed into every release. Thousands of products and services the world relies upon are built on FreeBSD, and the adoption rate of FreeBSD accelerates year after year. By all of these measures, FreeBSD is an amazingly successful open source project. But at the FreeBSD Foundation, we know today's success is just a step on a long path. We constantly ask, "How can we make FreeBSD even better?" "How can we take FreeBSD up that 'extra notch'?" Or as Nigel from 'This is Spinal Tap' would put it, "If we're playing maxed out at ten, how do we take it to eleven?"
Our answer for the 2013 year is to redefine the way the FreeBSD Foundation supports FreeBSD. Every year since I started the FreeBSD Foundation, we have grown our budget and activities by 10 to 25%, all managed by a single part-time employee. This next year we are investing in staff. Staff to bolster FreeBSD's amazing community of volunteers. Staff to scale the FreeBSD Foundation's funded development initiatives. Staff to double our capabilities in a single year.
This plan will increase the number and size of the development projects we fund. However, it is the FreeBSD Foundation's investment in "human infrastructure" that I believe will yield the largest dividends. As a volunteer myself, I know that no matter how deep my love for FreeBSD is, family and my "day job" must come first. Even with these other priorities, the volunteers that maintain the project's computer clusters, handle security incidents, tend our revision control and bug tracking systems, and generate our releases, show amazing skill and awe inspiring dedication. But our pool of volunteers for these areas hasn't grown as quickly as the rest of the project. By adding paid staff to support these roles, we will help reduce burn-out, provide continuity, and increase the effectiveness of FreeBSD's volunteers.
If things go as expected, by the end of 2013, the FreeBSD Foundation will have five employees, and an annual budget approaching $750,000. It will be a challenging transition, but a necessary step on the path of FreeBSD's continued success. Please consider giving us your support while we work to take FreeBSD all the way to eleven!
Justin T. Gibbs
President and Founder
The FreeBSD Foundation
Fundraising Update
Wow. I’ve been thoroughly overwhelmed by the outpouring of donations over the last few weeks! As of this publication we have raised $460,000 towards our goal of $500,000.
I want to thank you for everything you do to make this the best operating system around. There wouldn’t be a FreeBSD if we didn’t have you writing code, writing documentation, working on ports and releases, and educating current and future FreeBSD users.
We haven’t met our target yet, but we are getting close. Historically, each year, we have been half way to our fundraising goal at the start of December. Going forward, this is something that we plan on changing.
We unintentionally received some interesting press that disturbed a lot of FreeBSD users. This encouraged over 950 donations to come in, this past week. All I can say, is that it was incredible to see this support. One donor commented, “I don't use FreeBSD yet, but I've heard good things about the project, so that's why I wanted to support you.” How cool is that?
Your donations help us fund projects to improve FreeBSD, sponsor conferences and summits, purchase equipment to build infrastructure, promote FreeBSD, and provide legal counsel for the Project. In short, it helps us to provide the funding to make this the best OS available.
We’ve received some great lessons during this campaign. One thing we learned is that we need to advertise our fundraising needs outside of our FaceBook page, blog, and the FreeBSD Announce mailing list.
I hope that as you read through some of our accomplishments this year, you will consider making a donation to the Foundation. We can’t do it without you!
Efika SmartBook/SmartTop FreeBSD ARM Port
Tired of burning yourself with your laptop? If so, maybe it’s time for a change, such as switching to a platform that doesn’t require a large heat sink or loud cooling fans.
It is now possible to run FreeBSD on a low power, ARM-based, laptop: the Genesi Efika MX Smartbook. It has an ARM Cortex-A8 CPU inside, and, just like a cell phone, is energy efficient, consumes minimal power and uses passive cooling.
Our main project goal is to implement the missing pieces for the Freescale ARM SoC, including graphics, and to provide a great example of FreeBSD running on a low power device that developers can both use, and show off.
With the Efika MX code in the tree, we will have support for ARM-based laptops and even nettops (such as the Efika MX Smartop model) and we can continue pushing FreeBSD into ARM tablets and other ARM-based devices.
Capsicum Component Framework
Capsicum is a novel hybrid capability model that first appeared in FreeBSD 9.0, targeted at application compartmentalization: the mitigation of security vulnerabilities through decomposition of complex and risky applications into isolated components.
In the big picture, Capsicum provides a tight sandboxing mechanism where access to all global name spaces is restricted, and also allows actions that can be performed on file descriptors to be limited (file descriptors represent capabilities).
The purpose of my project was to make Capsicum more mature as well as easier to use by application developers.
The main part of the project was to create the Casper daemon, which is used by sandboxed processes to gain access to functionality that is not directly permitted within a sandbox. As an example, Casper provides a DNS service which can be use by sandboxed processes for host name resolution without the need to read /etc/resolv.conf or send any network packets to DNS servers.
Another part of the project was to remove capability wrapping descriptors - a special kind of file descriptors with a limited set of rights (e.g. a file descriptor which can be used only for reading, but not writing, fchmod, etc.). The capability wrapping descriptors were replaced with the ability to limit rights on any regular descriptor. This simplifies much of Capsicum's kernel support code and makes sandboxing more robust.
Initially the ioctl(2) system call was denied in capability mode (i.e. for sandboxed processes), because of its huge scope. Now, it is possible to limit the ioctl commands that can be performed within a sandbox, on a given descriptor. I also added the ability to control fcntl operations in a similar fashion.
Many bugs were fixed, many smaller improvements were made, and a lot of new regression tests were created along the way.
Capsicum is still on-going work, but thanks to funding from the FreeBSD Foundation and Google, it is maturing quickly and is starting to be used in real world applications (I personally use it in my other, FreeBSD Foundation sponsored, projects: HAST and auditdistd).
iSCSI Target
The goal of this project is to create a native, high performance, iSCSI target facility for FreeBSD. While configuration and connection setup and teardown are handled by a userland daemon, unlike previous target frameworks, all data-movement is performed in the kernel. The iSCSI target is fully integrated with the CAM Target Layer meaning that volumes can be backed by files or any block device. The hardware offload capabilities of modern network adapters will also be supported.
Much of the protocol and data movement logic can be shared between iSCSI target and initiator implementations. Once target support is robust, FreeBSD's existing iSCSI initiator will be updated to use many of the components developed for the target. This will improve initiator performance and add modern features such as InitialR2T.
The project is quickly progressing from a working prototype to a fully functional state; testing phase is expected to start in early January. After that the development will focus on implementing the hardware offload and performance tuning.
EuroBSDCon 2012
In October 2012, the 11th EuroBSDcon was held in the beautiful city of Warsaw in Poland. Four days of talking, learning about BSD, and of course exploring Polish culture (don't forget the Polish Hussars). The conference featured 2 tutorial days followed by two conference days on Saturday and Sunday. The conference consisted of a 2 track program, paralleled by a devsummit track on Saturday.
Traditionally, EuroBSDcon has a strong FreeBSD presence, but there is also enough room for the other BSD's. One of the reasons for the strong FreeBSD presence is the FreeBSD devsummit, which is held concurrently with the EuroBSDcon tutorial days.
The FreeBSD Foundation has been one of the regular sponsors over the years. This support plays an important role in making this an outstanding conference.
Besides sponsoring the conference, The FreeBSD Foundation also provides travel grants for visitors and developers who can not afford to go the conference on their own. This gives them the opportunity to visit the conference and exchange knowledge and ideas with other conference attendees, which greatly improves the value of having such conferences.
We are excited to announce that EuroBSDcon 2013 will be held September 28th-29th, 2013 (tutorials will be September 26th-27th) on Malta (St. Julian's area).
Cambridge Developer Summit 2012
Earlier this year, I had the pleasure of co-organizing the Cambridge DevSummit. Robert Watson, in contrast, had most of the stress, and others had the fun of running around and actually doing the work. This was a three-day event, with a documentation summit scheduled for the day before, organized primarily by Gavin Atkinson and Isabell Long.
This was the second such event hosted in Cambridge; the first in which no one fell in the river—possibly because the weather wasn't quite nice enough to encourage punting that week.
The three days of the main event were split into three sessions, with two tracks in each. I invited some visitors from ARM (who, conveniently, are just down the road) to attend one afternoon session. This was productive, and led to further engagement between the FreeBSD community and ARM. ARM is a customer-focused company, and those who may be interested in FreeBSD/ARM are potential customers of ARM's customers.
We finalized the schedule on the first day, filling an entire whiteboard with items that two or more people wanted to discuss and then splitting into groups. While a bit more advanced planning may have been helpful, our strategy worked relatively well this year. A short summary from each of the groups was presented in the final session and then published on the FreeBSD wiki.
The toolchain session, a big session, was of particular interest to me. We arrived at a tentative plan for throwing the switch to make clang the default compiler on FreeBSD-CURRENT. This was further discussed on the mailing list, and has now happened, bringing us one big step closer to a GPL-free FreeBSD 10.
An afternoon of short talks from researchers in the Cambridge Computer Lab involved either operating systems work in general or FreeBSD in particular. Robert Watson showed off a tablet running FreeBSD on a MIPS-compatible soft-core processor running on an Altera FPGA.
Cambridge, an old university town, predates the invention of hills and so is very amenable to cycling. We hired bikes for all non-local developers so they could get around easily.
The DevSummit dinner was hosted by St. John's College and cosponsored by Google and the FreeBSD Foundation. This enjoyable event, in the historic surroundings of a 500-year old college, provided a unique (and possibly never-to-be-repeated) opportunity of seeing a group of FreeBSD developers smartly dressed.
The day after the conference, Ollivier Robert organized a trip to Bletchley Park, which this year was celebrating Turing's centenary. Although it was an informative outing, I don't believe that it resulted in any concrete plans for a FreeBSD/Collossus port.
Bay Area Vendor Summit 2012
On November 8th The FreeBSD Foundation sponsored the FreeBSD Silicon Valley Vendor Summit. The summit was hosted by Yahoo at their Sunnyvale campus. We had over 50 participants from more than 20 different companies participate in the summit, which is a great turnout.
After a round of introductions we discussed the work that had been completed since the last summit, in May of 2012, including:
Altera IP core drivers
Amazon EC2 work
Comprehensive test framework (ATF)
DTrace on non-x86 (MIPS)
Intel graphics
Ivy Bridge hwpmc
clang/llvm MIPS
ZFS improvements
growfs on live fs
After going over the completed and still to be done items, we had a brief set of comments from Tom Hanrahan of Microsoft on HyperV support for FreeBSD. Microsoft's intention is to have full hypervisor support for FreeBSD and they are actively engaging with the community to get the proper drivers in the right places in our tree so that FreeBSD is a first class citizen in their system.
The first major presentation of the day was Jim Harris talking about VTune. VTune is a system to help developers understand the performance of their code. A small number of us have been working with Intel to get this software available on FreeBSD, as, at the moment, it is only available on Windows and Linux.
VTune includes two major components, a data collector and a visualization program. The collector has been ported to FreeBSD and is available as a pre-beta to interested developers. The visualization program continues to only be available on Windows and Linux, but since the collector and the visualizer are separate it is easy to collect information about programs, including the kernel, running on a FreeBSD host and display the results on another system. One feature that VTune continues to lack, but which Intel is working on, is support for call graphs. Right now the system can show you which line of code is causing a performance problem, which is quite useful, but it does not show the call graph, basically how your program happened to get to that point. This is a feature that Intel knows it needs on FreeBSD for the system to be completely accepted by vendors and other folks building high performance systems with FreeBSD. That being said, it's still an impressive piece of software and will definitely help people who care about performance tuning.
Jim says that those who are interested in using VTune in its current form should contact him directly to get ahold of the beta driver: Jim Harris . I've put up a "VTune How To" page on our Wiki and will be populating it over the next week.
After the morning break Adrian Chadd brought people up to date on what's going on with embedded systems and FreeBSD. There has been a good deal of progress here, in particular the work done recently on ARM processor support, with FreeBSD now running on several systems, including the BeagelBone, ShivaPlug, and the now ubiquitous Raspberry Pi. MIPS support continues to improve, with a significant chunk of work coming from the folks in Cambridge, as well as companies that are integrating FreeBSD on MIPS into their products. PowerPC is also moving along, although there remains a dearth of hardware on which to work. A good deal of discussion accompanied this section, in particular about what we need to do to get the rest of the system, i.e. ports and packages, up on these architectures. Here the story is also improving with a set of packages existing for ARM processors as well. People interested in this area should check out recent mail threads on arm@ and embedded@ as well as others.
After talking about embedded systems, Daichi Goto updated us on efforts with vendors in Japan. He and Hiroki Sato have been busy working with companies in Japan and have set up a BSD Consulting company which is working with large Japanese companies to build business services on top of FreeBSD, with a great deal of success. Other work they're doing includes getting a good deal of FreeBSD documentation and information translated into Japanese, so that it is more easily accessible to developers in Japan.
After a lunch break a new, at least to me, FreeBSD Vendor, CacheIQ gave a presentation about their product and their use of FreeBSD. CacheIQ builds an SSD based NFS caching system on top of FreeBSD and are interested in following the project's development tree more closely. What followed was a discussion of the project's release policies and best practices relating to developing a product with FreeBSD.
From CacheIQ we moved on to testing and ATF. Garrett Cooper presented the work he's been doing, in collaboration with Simon Gerraty and Marcel Moolenaar, to get ATF integrated into FreeBSD. ATF was originally written for NetBSD and has been in use there for a couple of years now. The goal is to have a good framework for automated testing in FreeBSD and to improve the quality of our testing. As anyone who has worked with test frameworks knows, there are many bike sheds to be avoided here, and Garrett, Simon and Marcel have worked quite hard to keep away from picking colors, and have instead gotten the system into shape and working on FreeBSD. [As of the writing of this newsletter ATF is now in HEAD under contrib/atf.]
Our last session of the day was creating a new have/need list for vendors with the twist this time that we added a "co sponsor" list. The FreeBSD Foundation is going to put up funds to co-sponsor development projects with companies working on FreeBSD. This has already been successfully done in the past with projects such as the NAND Flash project (co-sponsored with Juniper Networks) and others. The Foundation can't sponsor all these projects, but where there is an ability to get cooperation on funding the development it makes sense to get a few interested parties together to get the work done. The list of possible co-development projects includes:
NetConf Agent
EFI Boot on amd64
NDMP
MIPS Super Pages
SID Base Credentials and ACLs
Xen Dom 0
Upcoming summits are in March at AsiaBSDCon and May at BSDCan. Mark your calendars.
2012 Grant and Travel Grant Recipients
Every year we sponsor FreeBSD related conferences and travel to these events for FreeBSD contributors. We believe that BSD-centered and FreeBSD-specific conferences play the dual roles of expanding the FreeBSD user community and supporting collaborative development. The FreeBSD Foundation's travel grant program helps to reduce financial roadblocks to participation in these events.
Our grant recipients often send us amazing tales of their experiences, proving the value of this program to the FreeBSD community. You can find these stories and trip reports on our blog.
Here is a list of projects, developers, and conferences we have sponsored for 2012.
2012 Conference Grant Recipients:
AsiaBSDCon 2012 Conference
BSDCan 2012 Conference
Ottawa 2012 Developer Summit
Ottawa 2012 Vendor Summit
BSDDay 2012
EuroBSDCon 2012 Conference
Cambridge 2012 Developer Summit
Bay Area 2012 Vendor Summit
2012 Project Grant Recipients:
Edward Napierala - iSCSI Target project
Pawel Jakub Dawidek - Capsicum Component Framework
Edward Napierala - Growing Filesystmes Online
Björn Zeeb - IPv6 Performance Analysis
Pawel Jakub Dawidek - Implementing auditdistd
Semihalf - NAND Flash Support
Aleksandr Rybalko - Porting FreeBSD to Efika ARM platform
2012 Travel Grant Recipients:
BSDCan - Hiren Panchasara, Adrian Chadd, Florian Smeets, Ben Haga, Marius Strobl, Brooks Davis, Julien Laffaye, Warren Block, Daichi Goto, Giovanni Trematerra, Davide Italiano, Thomas Abthorpe
EuroBSDCon - Gabor Pali, Alberto Mijares, Gabor Kovesdan, Alexander Pronin
Open Help - Warren Block
MeetBSD - Mark Linimon
Faces of FreeBSD Series
Who are these people that receive money from the Foundation? How are your donations making a difference in the FreeBSD community? We've asked our grant recipients to share their stories with you in what we call the "Faces of FreeBSD" campaign. Tune in weekly to the FreeBSD Foundations's blog to hear how Foundation funding is used to run conferences, work on development projects, help with travel expenses to conferences, and to advocate for FreeBSD.
Looking for a recap of past stories? Here are our first two "Faces of FreeBSD":
Faces of FreeBSD - Dan Langille
Faces of FreeBSD - Alberto Mijares
Netflix
More than 30 million Netflix streaming members around the globe watch more than a billion hours of movies and TV shows each month. In the US, Netflix video streaming accounts for a third of peak downstream Internet traffic. Netflix created Open Connect, a single-purpose content delivery network, to help deliver these petabytes of data.
The main component of Open Connect is the Open Connect Appliance, a small-footprint network streaming device. The Open Connect Appliance is a 4U Intel-based server that is designed to economically maximize storage density in a space and power footprint that is ideal for both ISP data centers and metro-area network interchanges.
The Open Connect Appliance runs on FreeBSD 9. Netflix picked FreeBSD 9 because it is a high performing, low-maintenance and reliable operating system that is supported by major hardware vendors. FreeBSD 9 provides a foundation of reliability, performance, and hands-free manageability. Combined with NGINX, a light-weight and high performance Web server, FreeBSD 9 provides a simple but powerful solution that is capable of serving tens of thousands of simultaneous video streams across multiple 10Gbit fiber optic links.
Beyond its technical strengths, FreeBSD comes with an outstanding ecosystem of developers, vendors, and users who openly share expertise, talent, and technical improvements. Netflix has embraced this community and is committed to giving back its bug fixes and enhancements, thus completing the circle of community collaboration.
- David Fullagar, Director of Content Delivery Architecture, Netflix
Friday, December 21, 2012
FreeBSD Foundation Newsletter
Monday, December 17, 2012
Faces of FreeBSD - Dan Langille
Sunday, December 16, 2012
MeetBSD Trip Report: Mark Linimon
The Foundation sponsored Mark Linimon's travel to California from October 28th to November 9th, 2012, to attend MeetBSD, the preceding Developer's Summit, the postceding Vendor Summit, and to perform work on the FreeBSD systems at ISC. Here is his trip report:
Developer Summit:
I gave a brief overview of the state of the Ports Collection, and invited those interested in more details to attend my talk at MeetBSD proper.
In addition, I answered a number of question in informal sessions from various developers and contributors, some of whom I had met before, and some whom I had not.
Among the issues brought up were ports on various architectures, cross-building ports or perhaps building ports under emulation, and plans for Content Distribution Networks.
Of particular note, I was able to interact in person with the Yahoo team (Peter Wemm, Sean Bruno, and Ben Haga) to talk about issues affecting our machines in the cluster.
MeetBSD:
I presented a session about the state of the Ports Collection on FreeBSD, and future directions. You may view my slides online.
In particular, I focused on the progress in our new packaging system pkgng; our reworked options system optionsng; the state of ports vs. clang; and the Redports distributed build testing system. It should be noted that hardware for both production package building and also Redports was purchased via Foundation funds.
I also attended other sessions that I found useful, especially Sean Bruno's explanation of qemu. I hope that we will be able to use this for cross-building ports in the future.
Vendor Summit:
At the Vendor Summit mostly I was an observer. However, I did present a brief explanation of the status of the ports tree, and how that might affect the vendors. As well, I chatted informally with Tom Hanrahan of Microsoft's Hyper-V effort about how best to interact with the FreeBSD community. In addition, I was able to interact with a number of other representatives from vendors who had questions about the Ports Collection, particularly about building for non-x86 architectures.
ISC:
The ISC systems needed some major work.
For one day overlapping my trip, Simon Nielsen was also on-site and did the preliminary install of 7 new Atom servers. These were purchased by the Foundation to be administration servers for a future fallover facility for the main Yahoo systems. I continued this work and did other work as follows:
- I moved most of the Ethernet cables from the old, unmanaged, switches to the Juniper EX3300 managed switch that Simon had also installed.
- I wired serial cables to the new Atom servers and verified that they worked properly.
- I returned one of the Atom servers to iXsystems for an RMA. It turned out that the CPU had suffered early failure. I reinstalled the system once they were finished.
- We still had several systems in rack B2, which ISC wishes us to vacate. Since the portmgr-owened systems there were offline due to other reasons, I relocated them. Two systems still remain.
- I deinstalled a power controller that had failed.
- The cable situation had become sub-optimal. In many cases, the only cables that had been on hand were 25' or more. It was barely possible to get to the back of the systems, and airflow was a problem. I installed various shorter lengths of cable, and correctly color-coded them while doing so.
- I verified all the connections to the console server and the PDUs.
- I attempted to install a new hard drive on our PowerPC server there. However, without a CDROM drive, that system is hard to work with. Instead, I carried the hard drive back to Austin, where I will format it up on my similar system here and send it back to ISC.
- I purchased a second power supply for the Coverity machine.
- I removed rack slides from the systems on which those slides do not fit into the shallow racks there.
- I purchased some rack shelves that will stand in for these slides to help us manage our physical space. They still need to be installed.
- I physically labeled all the systems.
- I annotated all the above in our administration database.
Wednesday, December 12, 2012
*Stunning News Website Fundraising Contribution:* Over 650 new donations raise $43,200 in three days!
Astute readers of our blog know that The FreeBSD Foundation's annual year-end fundraising drive began last week. Every year over 50% of our donations arrive during this campaign. We were thus surprised by coverage on Slashdot and Hacker News suggesting that we were behind in our fundraising goal. We’d never turn down additional news coverage of our work raising funds in support of the FreeBSD project, of course, but remain confident in our fundraising target.
The response to those articles was a remarkable outpouring of support — we have had over 650 new donations in the last three days alone, raising $43,196. Individual donations are absolutely critical to the business of The Foundation. Not only do they contribute significantly to our ability to financially support the FreeBSD project, but they also help us demonstrate broad public support in order to maintain our public non-profit status.
This massive increase in small donations will set a new record for individual donors in a single year. The readership of these sites has been instrumental to this success. You'll also be glad to know that the news article triggered a number of further corporate contributions from FreeBSD consumers. Our hard work is not done, however. We still have $195,000 left to go to meet our target this year–or exceed it!
Look forward to more news soon. . .