Letter from the President

From the December Foundation Newsletter:

In 2009, the FreeBSD project had the misfortune of losing two long time contributors: John Birrell and Jean-Marc Zucconi. I chatted with John recently, during this year's BSDCAN, so his death was all the more shocking. It forced me to recognize my own mortality and to consider what contributions from our lives remain after we pass away. Reviewing the heritage of FreeBSD it becomes clear that our work on this project takes on a life of its own. John and Jean-Marc's efforts live on in FreeBSD.

The value of the FreeBSD legacy has become even more apparent to me with my return, after an almost 10 year break, to working on FreeBSD during my day job. The kernel has the familiarity of an old friend, even though it has been improved in countless ways by dozens of new contributors. Although the principles and best practices engendered by the FreeBSD project have changed little in the last decade, they are no less relevant today. The faces and challenges may have changed, but the qualities of FreeBSD - solid design, high performance, stability - have not.

Trying to communicate this real value of FreeBSD can be difficult. Valuations typically take current features into account, but neglect to consider the community that created and will support those features. And what value can you place on future, community driven, improvements to FreeBSD? There is no guarantee that every feature you need will be added to the system in the time frame you need it, but the collaborative environment created by the FreeBSD project makes it very likely. The greatest asset of FreeBSD is our community. If we continue to invest in the FreeBSD community, I have every confidence that the FreeBSD legacy will endure.

The FreeBSD Foundation is doing its part to continue FreeBSD's legacy. Through conservative management and careful planning we have improved our financial position even when faced with the most severe recession in 50 years. Our growing strength and capabilities, all made possible by your generous donations, are proof that the FreeBSD Foundation will be a faithful supporter of FreeBSD for many years to come. Together, building on the legacy left by John, Jean-Marc, and the countless contributors before them, we will ensure the future of FreeBSD.

FreeBSD Project Receives Bad Code Offset

The Bad Code Offsets project of The Alliance for Code Excellence is "a way to undo the bad code other people have written without actually replacing the bad code. Much like carbon offsets, money used to buy Bad Code Offsets goes towards open-source projects which not only produce good code, but produce software that helps developers build good software".

The FreeBSD Project was recently added as a supported project and the FreeBSD Foundation will be receiving a check for $500. Thanks to those who suggested the FreeBSD project be added and to the Alliance for supporting good code!

Google Summer of Code Mentor Summit 2009

Brooks Davis recently reported on his trip to the Google Summer of Code Mentor Summit 2009. The Foundation assisted in some of his travel costs to this event. Brook writes:

The Google Summer of Code Mentor Summit was held at the Google campus in Mountain View October 24th and 25th. I represented the FreeBSD project at the event along with Tim Kientzle who was one of our other program admins.

Google runs the Mentor Summit as an un-conference which means that attendees pick topics they want to discuss, others indicate interest in them, and then rooms are allocated based on demand. Sessions were about things including general open source process issues, education, technical collaborations, and individual project meetings.

One of the highlights of Saturday was a session on multi-core and other acceleration technologies like GPUs. The session didn't come to a strong consensus other than incorporating these technologies is difficult. The most concrete thing that came up was the idea of putting the technologies behind widely used APIs so they automatically provide benefit. Another topic of discussion was debugging tools and
techniques. I think that is an area where FreeBSD is sometimes ahead with technologies like witness and now DTrace.

The other session of interest was an impromptu session of non-Linux OSes including DragonFlyBSD, FreeBSD, Haiku, and NetBSD. It was mostly people talking about current status. One thing other projects seemed to want was a way to take advantage of FreeBSD network drivers by providing more common interfaces. In concept this was interesting, but probably isn't some thing that would make a whole lot of sense for FreeBSD given that we're rethinking and redesigning the interfaces we have to meet modern performance requirements.

The most useful session on Sunday was when Tim and I grabbed a small conference room and started reviewing our GSoC admin materials for next year. Based on that information, we plan to start engaging with FreeBSD developers in January in anticipation of a 2010 program.

Over all the conference was interesting and fun. It was good to talk to
people from other projects that don't often attend the BSD conference.
I'm not sure anything concrete came of those interactions, but it was
probably useful nonetheless.

Thank You!

We would like to thank everyone who has donated to the FreeBSD Foundation this year. We have raised $182,778 towards our 2009 goal of $300,000! We are almost 2/3 of the way to reaching our goal! Oh, and BTW, we have had 663 donors this year. This is compared to just over 300 this time last year. This is important not only to help us keep our Public Charity Status, but it shows there are many users who are passionate about FreeBSD and want to show their support.

With the weakened economy we have been very conservative with our spending this year. But, like each previous year we have increased the amount we have spent on the FreeBSD Project and community. We were blown away with the number of project proposals we received this year. We were able to fund 7 projects this year. Unfortunately we didn't have the budget to fund all the proposals we received.

This coming year we want to double the amount we spend on project development. In order to accomplish this, we need to meet our fund-raising goal.

Why do we need donations?

The goal of the FreeBSD Project is to provide software that may be used for any purpose -- and without strings attached. Our mission is to support the FreeBSD Project and community. Our funding comes from people like you – those who are determined to keep FreeBSD free!

How have we spent the money this year?

• Sponsored FreeBSD related conferences like BSDCan, EuroBSDCon, AsiaBSDCon, KyivBSD, and DCBSDCon.

• Provided 15 travel grants and funding to individuals to attend these and other conferences this year.

• Provided grants for projects that improve FreeBSD, like wireless mesh support, FreeBSD TCP stack improvements, new console driver, safe removal of disk devices, flattened device tree, and high available storage projects.

• Provided equipment for developers working to improve FreeBSD and projects like the NetPerf cluster. We purchased servers, USB analyzers, power controllers, and parts for computer repairs for the Project. We also paid for shipping of equipment to various projects.

• Provided legal support for the project on issues like understanding the GPLv3 impact on FreeBSD, providing a privacy policy, trademark ownership and permission, and other legal issues that come up.

How can you help?

Your financial support is critical for the FreeBSD Project. Please help us keep FreeBSD free. Any amount you can donate will help. And thank you for your continued support of the FreeBSD Foundation.

Update on HAST Project

Pawel Jakub Dawidek has completed the first milestone for the High Available Storage Project. He writes:

I want to report that first milestone of the HAST project is complete.

Summary of the work that have been done:

• Implementation of hastd daemon.
• Implementation of hastctl utility to manage hastd daemons.
• GEOM_GATE class was extended so that the caller can specify the name of GEOM provider. Before only /dev/ggateX names were supported. HAST will use /dev/hast/<resource_name> names.
• Implementation of communication protocol. There is abstraction layer on top and below there are three protocols implemented currently:
  

1. proto_tcp4 - It is used for communication between primary and secondary nodes.
2. proto_uds - (UDS - UNIX Domain Socket) It is used for communication between hastctl and hastd.
   
3. proto_socketpair - It is used for communication between main hastd daemon and worker processes forked from it.

• Implementation of nv (name-value) API, which allows to easy create packets containing name-value pairs. It is used for entire communication through the protocols above. It is also responsible for managing correct byte-order.
• Implementation of ebuf (extendable buffer) API, which provides a way to extend given buffer by adding data at the back, but also at the front without reallocating it and copying the data very often (or never).
• Implementation of logging API (pjdlog). The API decides if messages should be logged on standard output/error (before going into background) or to syslog (when we daemonize). It also provides some shortcuts for logging a message and exiting, etc. It supports notion of debug level and can skip messages intended for higher debug level than requested.
• Implementation of configuration file parser in lex/yacc. Configuration file is designed in a way that it can be kept identical on both nodes.
• Checksumming and compression for the data is not one of the project's goal, but the stubs are there, so this can be added easly.
• A lot of care was taken to be able to handle more nodes in the future. This is not implemented and in not project goal, but I wanted to make it ready for future improvements.
  

HAST can be used by starting hastd daemons on both nodes:

# hastd -h
hastd: [-dh] [-c config]

Then on the secondary node we mark all resources as secondary:

# hastctl -h
hastctl: [-d] [-c config]
hastctl: [-d] [-c config] status [all | name ...]

# hastctl secondary all

On the primary node we mark all resources as primary:

# hastctl primary all

The hastd daemon running on primary node will connect to the secondary node and fork a child to handle communication. There is a socketpair between parent and child so that they can communicate. Primary node creates two connections: one for incoming data and one for outgoing data. There are seven threads in total for each working resource:

1. ggate_recv

Thread receives ggate I/O requests from the kernel and passes them to appropriate threads:
WRITE - always goes to both local_send and remote_send threads
READ (when the block is up-to-date on local component) -only local_send thread
READ (when the block isn't up-to-date on local component) -only remote_send thread
DELETE - always goes to both local_send and remote_send threads
FLUSH - always goes to both local_send and remote_send threads

2. local_send

Thread reads from or writes to local component.
If local read fails, it redirects it to remote_send thread.

3. remote_send

Thread sends request to secondary node.

4. remote_recv

Thread receives answer from secondary node and passes it to ggate_send thread.

5. ggate_send

Thread sends answer to the kernel.

6. ctrl

Thread handles control requests from the parent.

7. guard

Thread guards remote connections and reconnects when needed, handles signals, etc.

On the secondary node when both connections are successfully established it forks a worker process, which operates using four threads:

1. recv

Thread receives requests from the primary node.

2. disk

Thread reads from or writes to local component and also handles DELETE and FLUSH requests.

3. send

Thread sends requests back to primary node.

4. ctrl

Thread handles control requests from the parent.

At this point primary and secondary nodes can communicate and requests are properly replicated. IO errors on local read failure are handled by redirecting read request to remote node. Replicated storage can be accessed through /dev/hast/<resource_name> GEOM provider.

I'm confident that the first milestone is complete. If you have any questions, I'll be happy to answer them. If you have any suggestions or comments, I'll also be happy to hear them.

Colin Percival on Supporting the Foundation

Developers not only benefit from the work done by the Foundation, they also help to financially support it. Colin Percival writes about donating to the Foundation on his blog. His blog post begins:

As a FreeBSD user and developer, I obviously care about the success of FreeBSD. I make a small contribution towards this success via my role as Security Officer; but the time I spend working on my Tarsnap online backup service prevents me from making as much of a direct contribution as I would like. Fortunately the FreeBSD Foundation does an excellent job of supporting FreeBSD development; but like most such organizations, they are funded entirely by donations and are always in need of more. In light of this, I am pleased to announce that I will be donating all of the profits made by Tarsnap for the month of December to the FreeBSD Foundation.

Much of the work done by the FreeBSD Foundation is done behind the scenes, but the importance of the work they do is undeniable. They sponsor a range of important FreeBSD development which would likely not get done otherwise; in the past year this has included...

In Memory of Donations

It is with sadness that we note the passing of long time FreeBSD committer, John Birrell. Many in the FreeBSD community had the opportunity to interact with John online and perhaps meet him in person at a BSDCan. He will be missed.

It is now possible to make a donation to the FreeBSD Foundation in memory of someone. Select the "in memory of" button on the donation page and type in the name of the person the donation is being made in memory of.