Recently posted on the Facebook blog:
Almost two million new users from around the world sign up for Facebook each week—and we couldn’t be happier. It’s tremendously rewarding to see so many people find what we work on useful and fun. As we continue to add new users and features, however, the load on our thousands of servers continues to increase at a pretty astounding rate. A few weeks ago we reached full capacity in our California datacenters. In the past we handled this problem by purchasing a few dozen servers, hooking them up, and getting on with our lives, but this time we didn’t have it so easy. We’d actually run out of space in our datacenters for new machines.
Fortunately we saw this problem coming a long time ago and started work on a new datacenter in Virginia. Now, we identify whether a user would be better off talking to the east coast datacenter or a west coast data center. For people in Europe and the eastern half of the US, it’s noticeably faster to talk to a server in Virginia than in California. For these users we direct them to Virginia whenever they’re browsing the site and not making any changes.
Whenever that person goes to change some data—uploading a photo album, or changing profile info for example—we send them off to California so that all our modifying operations happen in the same location. This decision was made to prevent two or more modifications from conflicting with each other and messing up our data. It might sound like we’re forcing our users to go to California a lot but only about 10% of our traffic causes a modifying operation. MySQL has a great replication feature that allows us to, in real time, stream all the modifications happening on a California MySQL server to another one in Virginia. Replication happens so fast, even across the country, that the Virginia servers are almost never more than one or two seconds behind the California servers.
Even though all of the modification happens in California and streams instantly to Virginia, we were faced with another problem. Although Facebook’s data is stored in MySQL database servers, we use a large number of memcached servers to store copies of the data. Memcached is much faster and able to keep up with requests quicker than the databases themselves can keep up. We had to figure out a way for memcached servers to replicate data concurrently with the MySQL databases. Because of various technical limitations of our architecture there was no easy way to do so.
Fortunately MySQL is open source software, meaning we can actually change the way it works by modifying the code. We did just that—embedding extra information in to the MySQL replication stream that allows us to properly update memcached in Virginia. This ensures that the cache and the database are always in sync. Over the last seven months a great team of Facebook employees has been building new software and setting up new servers like I described above. Over Thanksgiving we finally flipped the switch and since then almost 30% of our traffic has been served from Virginia.
The east coast datacenter is a great first step towards keeping Facebook fast and reliable as the site grows. Going forward we have lots of exciting plans to expand our infrastructure and improve performance so no user ever has to sit around waiting for a page to load.
Read the rest of this entry »
Posted in Facebook, Internet | No Comments »
The US defence department has banned the giant internet search engine Google from filming inside and making detailed studies of US military bases.
Close-up, ground-level imagery of US military sites posed a “potential threat” to security, it said.
The move follows the discovery of images of the Fort Sam Houston army base in Texas on Google Maps.
A Google spokesman said that where the US military had expressed concerns, images had been removed.
Google has now been barred from filming and conducting detailed studies of bases, following the discovery of detailed, three-dimensional panoramas online – and in particular, views of the Texan base.
“Images include 360-degree views of the covered area to include access control points, barriers, headquarters, facilities and community areas,” said the defence department in a statement quoted by AFP news agency.
It said such detailed mapping could pose a threat.
Google spokesman Larry Yu said the decision by a Google team to enter the Texas base, which is in San Antonio, and undertake a detailed survey, had been “a mistake”.
He told the BBC that it was “not our policy to request access to military installations, but in this instance the operator of the vehicle with the camera on top – which is how we go about capturing imagery for Street-View – requested permission to access a military installation, was given access, and after learning of the incident we quickly removed the imagery”.
Individuals and governments
Military officials are currently looking into exactly what imagery is available – though it may not be able to order its removal if images are taken from public streets.
Among the popular mapping services offered by Google are Street View, which allows web users to “drive” along virtual US landscapes with ground-level views, and Google Earth, which offers detailed satellite and 3D images of locations around the world.
In this case, it was imagery offered on Street View that caused the concern.
But both have provoked complaints – from individuals depicted in the images and from governments concerned that satellite images could compromise security.
Gary Ross, a spokesman for the US Northern Command, told AFP that although such services could be useful, “there has to be a balance”.
But Mr Yu said Google would listen to concerns about privacy and security.
“We try to have a compliant image removal policy – not only relative to the military but to consumers also,” said Mr Yu.
“If people have concerns, they should contact us.”
Read the rest of this entry »
Posted in Google, Internet | No Comments »
Microsoft showed off two social-networking projects at TechFest on Tuesday that show that the company wants to do more in this area than just invest in Facebook.
One project, known as Salsa, aims to use one’s corporate data to piece together their social network, or at least their network of co-workers. In its current form, the software is a plug-in to Outlook that shows social-networking information such as a photo and profile next to an incoming e-mail message. The program also pieces together a list of “friends” based on e-mail frequency and other data.
“When you start looking there is a surprising amount of information that gets locked in e-mail,” said Shane Williams, one of the Microsoft Research team that worked on Salsa.
Lili Cheng, the Microsoft veteran who heads the social-computing team at Microsoft Research, said that part of the power of Salsa is simply putting a human face on e-mail. She said her own use of the site has borne out the power of that, noting it is harder to argue with a colleague when she sees a picture of them with their cute kid or pet.
“E-mail can be very dehumanizing,” Cheng said.
Cheng said that in addition to deploying it inside Microsoft, she’d like to see how Salsa works within one or two other large companies to see if it is more broadly useful.
In another project from Cheng’s group, known as C2, Microsoft researchers have created a Windows application that pieces together contact data from a variety of social-networking sites. For the purposes of Tuesday’s demonstration, the researchers focused on Windows Live Spaces and Facebook. Researcher Steve Ickman said he chose those two because they represent among the most open (Spaces) and closed (Facebook) when it comes to data sharing.
Although Facebook is notoriously restrictive when it comes to members scraping their data, Ickman said that he believes he was able to stay within Facebook’s terms of service by grabbing only approved data from one’s own contacts and not caching the information long-term. “It’s totally legal, at least at this point.”
The project is more of a technology demonstration than anything geared toward a specific product, Ickman said, adding that he hoped it would demonstrate to the product teams that they can be more ambitious. “We tend to cancel things because they are too hard,” he said.
Read the rest of this entry »
Posted in Internet, Microsoft, Web 2.0 | No Comments »
