又遇到BCM5709网卡问题，现在整理下，下面是升级驱动的方法：
ln -s /usr/src/kernels/2.6.18-194.26.1.el5-x86_64 /usr/src/kernels/2.6.18-194.el5-x86_64
ln -s /lib/modules/2.6.18-194.el5 /lib/modules/2.6.18-194.26.1.el5
tar zxvfp Bcom_LAN_14.1.5_Linux_Source_A00.tar.gz
cd Bcom_LAN_14.1.5_Linux_Source_A00/NetXtremeII/
tar zxvfp netxtreme2-5.0a.10.tar.gz
cd netxtreme2-5.0a.10
make
make KVER=2.6.18-194.el5
make install
重启机器生效或者重新加载bnx2驱动：
rmmod bnx2
modinfo bnx2
It is widely accepted that fast-loading pages improve the user experience. In recent years, many sites have started using AJAX techniques to reduce latency. Rather than round-trip through the server retrieving a completely new page with every click, often the browser can either alter the layout of the page instantly or fetch a small amount of HTML, XML, or javascript from the server and alter the existing page. In either case, this significantly decreases the amount of time between a user click and the browser finishing rendering the new content.
However, for many sites that reference dozens of external objects, the majority of the page load time is spent in separate HTTP requests for images, javascript, and stylesheets. AJAX probably could help, but speeding up or eliminating these separate HTTP requests might help more, yet there isn’t a common body of knowledge about how to do so.
While working on optimizing page load times for a high-profile AJAX application, I had a chance to investigate how much I could reduce latency due to external objects. Specifically, I looked into how the HTTP client implementation in common browsers and characteristics of common Internet connections affect page load time for pages with many small objects.
I found a few things to be interesting:
Using these, I came up with a model to guesstimate the effective bandwidth of users of various flavors of network connections when loading various object sizes. It assumes that each HTTP request is 500 bytes and that the HTTP reply includes 500 bytes of headers in addition to the object requested. It is simplistic and only covers connection limits and asymmetric bandwidth, and doesn’t account for the TCP handshake of the first request of a persistent (keepalive) connection, which is amortized when requesting many objects from the same connection. Note that this is best-case effective bandwidth and doesn’t include other limitations like TCP slow-start, packet loss, etc. The results are interesting enough to suggest avenues of exploration but are no substitute for actually measuring the difference with real browsers.
To show the effect of keepalives and multiple hostnames, I simulated a user on net offering 1.5Mbit down/384Kbit up who is 100ms away with 0% packet loss. This roughly corresponds to medium-speed ADSL on the other side of the U.S. from your servers. Shown here is the effective bandwidth while loading a page with many objects of a given size, with effective bandwidth defined as total object bytes received divided by the time to receive them:
![[1.5megabit 100ms graph]](http://www.die.net/musings/page_load_time/ebps_1500k_100ms.png)
Interesting things to note:
Perhaps more clearly, the following is a graph of how much faster pages could load for an assortment of common access speeds and latencies with many external objects spread over four hostnames and keepalives enabled. Baseline (0%) is one hostname and keepalives disabled.
![[Speedup of 4 hostnames and keepalives on]](http://www.die.net/musings/page_load_time/ebps_speedup.png)
Interesting things from that graph:
One more thing I examined was the effect of request size on effective bandwidth. The above graphs assumed 500 byte requests and 500 bytes of reply headers in addition to the object contents. How does changing that affect performance of our 1.5Mbit down/384Kbit up and 100ms away user, assuming we’re already using four hostnames and keepalives?
![[Effective bandwidth at various request sizes]](http://www.die.net/musings/page_load_time/ebps_request_size.png)
This shows that at small object sizes, we’re bottlenecked on the upstream bandwidth. The browser sending larger requests (such as ones laden with lots of cookies) seems to slow the requests down by 40% worst-case for this user.
As I’ve said, these graphs are based on a simulation and don’t account for a number of real-world factors. But I’ve unscientifically verified the results with real browsers on real net and believe them to be a useful gauge. I’d like to find the time and resources to reproduce these using real data collected from real browsers over a range of object sizes, access speeds, and latencies.
You can measure the effective bandwidth of your users on your site relatively easily, and if the effective bandwidth of users viewing your pages is substantially below their available downstream bandwidth, it might be worth attempting to improve this.
Before giving the browser any external object references (<img src=”…”>, <link rel=”stylesheet” href=”…”>, <script src=”…”>, etc), record the current time. After the page load is done, subtract the time you began, and include that time in the URL of an image you reference off of your server.
Sample javascript implementing this:
<html>
<head>
<title>...</title>
<script type="text/javascript">
<!--
var began_loading = (new Date()).getTime();
function done_loading() {
(new Image()).src = '/timer.gif?u=' + self.location + '&t=' +
(((new Date()).getTime() - began_loading) / 1000);
}
// -->
</script>
<!--
Reference any external javascript or stylesheets after the above block.
// -->
</head>
<body onload="done_loading()">
<!--
Put your normal page content here.
// -->
</body>
</html>
This will produce web log entries of the form:
10.1.2.3 - - [28/Oct/2006:13:47:45 -0700] "GET /timer.gif?u=http://example.com/page.html&t=0.971 HTTP/1.1" 200 49 ...
in this case, showing that for this user, loading the rest of http://example.com/page.html took 0.971 seconds. And if you know that the combined size of everything referenced from that page is 57842 bytes, 57842 bytes * 8 bits per byte / 0.971 seconds = 476556 bits per second effective bandwidth for that page load. If this user should be getting 1.5Mbit downstream bandwidth, there is substantial room for improvement.
After you gather some page-load times and effective bandwidth for real users all over the world, you can experiment with changes that will improve those times. Measure the difference and keep any that offer a substantial improvement.
Try some of the following:
When you generate a page, evenly spreading your images over four hostnames is most easily done with a hash function, like MD5. Rather than having all <img> tags load objects from http://static.example.com/, create four hostnames (e.g. static0.example.com, static1.example.com, static2.example.com, static3.example.com) and use two bits from an MD5 of the image path to choose which of the four hosts you reference in the <img> tag. Make sure all pages consistently reference the same hostname for the same image URL, or you’ll end up defeating caching.
Beware that each additional hostname adds the overhead of an extra DNS lookup and an extra TCP three-way handshake. If your users have pipelining enabled or a given page loads fewer than around a dozen objects, they will see no benefit from the increased concurrency and the site may actually load more slowly. The benefits only become apparent on pages with larger numbers of objects. Be sure to measure the difference seen by your users if you implement this.
Set an Expires header on everything you can, with a date days or even months into the future. This tells the browser it is okay to not revalidate on every request, which can add latency of at least one round-trip per object per page load for no reason.
Instead of relying on the browser to revalidate its cache, if you change an object, change its URL. One simple way to do this for static objects if you have staged pushes is to have the push process create a new directory named by the build number, and teach your site to always reference objects out of the current build’s base URL. (Instead of <img src=”http://example.com/logo.gif”> you’d use <img src=”http://example.com/build/1234/logo.gif”>. When you do another build next week, all references change to <img src=”http://example.com/build/1235/logo.gif”>.) This also nicely solves problems with browsers sometimes caching things longer than they should — since the URL changed, they think it is a completely different object.
If you conditionally gzip HTML, javascript, or CSS, you probably want to add a “Cache-Control: private” if you set an Expires header. This will prevent problems with caching by proxies that won’t understand that your gzipped content can’t be served to everyone. (The Vary header was designed to do this more elegantly, but you can’t use it because of IE brokenness.)
For anything where you always serve the exact same content when given the same URL (e.g. static images), add “Cache-Control: public” to give proxies explicit permission to cache the result and serve it to different users. If a caching proxy local to the user has the content, it is likely to have much less latency than you; why not let it serve your static objects if it has them?
Avoid the use of query params in image URLs, etc. At least the Squid cache refuses to cache any URL containing a question mark by default. I’ve heard rumors that other things won’t cache those URLs at all, but I don’t have more information.
When serving a static files (including HTML) off of disk, most webservers will generate Last-Modified and/or ETag reply headers for you and make use of the correspondingIf-Modified-Since and/or If-None-Match mechanisms on requests. But as soon as you add server-side includes, dynamic templating, or have code generating your content as it is served, you are usually on your own to implement these.
The idea is pretty simple: When you generate a page, you give the browser a little extra information about exactly what was on the page you sent. When the browser asks for the same page again, it gives you this information back. If it matches what you were going to send, you know that the browser already has a copy and send a much smaller 304 (Not Modified) reply instead of the contents of the page again. And if you are clever about what information you include in an ETag, you can usually skip the most expensive database queries that would’ve gone into generating the page.
If you are brave, you could also try to figure out which set of browsers will handle compressed Javascript properly. (Hint: IE4 through IE6 asks for its javascript compressed, then breaks badly if you send it that way.) Or look into Javascript obfuscators that strip out whitespace, comments, etc and usually get it down to 1/3 to 1/2 its original size.
To implement the slow proxy, I used the netem and HTB kernel modules available in the Linux 2.6 kernel, both of which are set up with the tc command line tool. These offer the most accurate simulation I could find, but are definitely not for the faint of heart. I’ve not used them, but supposedly Tamper Data for Firefox, Fiddler for Windows, and Charles for OSX can all rate-limit and are probably easier to set up, but they may not simulate latency properly.
The Safari team offers a tip on a hidden feature in their browser that offers some timing data too.
Or if you are familiar with the HTTP protocol and TCP/IP at the packet level, you can watch what is going on using tcpdump, ngrep, or ethereal. These tools are indispensable for all sorts of network debugging.
If your latencies are high and your webserver process (or CGI if you are using that) is eating a lot of CPU during this test, it is often a result of using a scripting language that needs to recompile your scripts with every request. Software like eAccelerator for PHP, mod_perl for perl, mod_python for python, etc can cache your scripts in a compiled state, dramatically speeding up your site. Beyond that, look at finding a profiler for your language that can tell you where you are spending your CPU. If you improve that, your pages will load faster and you’ll be able to handle more traffic with fewer machines.
If your site relies on doing a lot of database work or some other time-consuming task to generate the page, consider adding server-side caching of the slow operation. Most people start with writing a cache to local memory or local disk, but that starts to fall down if you expand to more than a few web server machines. Look into usingmemcached, which essentially creates an extremely fast shared cache that’s the combined size of the spare RAM you give it off of all of your machines. It has clients available in most common languages.
The above list covers improving the speed of communication between browser and server and can be applied generally to many sites, regardless of what web server software they use or what language the code behind your site is written in. There is, unfortunately, a lot that isn’t covered.
While the tips above are intended to improve your page load times, a side benefit of many of them is a reduction in server bandwidth and CPU needed for the average page view. Reducing your costs while improving your user experience seems it should be worth spending some time on.
Linux中查看socket状态：
cat /proc/net/sockstat #（这个是ipv4的）
sockets: used 137 TCP: inuse 49 orphan 0 tw 3272 alloc 52 mem 46 UDP: inuse 1 mem 0 RAW: inuse 0 FRAG: inuse 0 memory 0
说明：
sockets: used：已使用的所有协议套接字总量
TCP: inuse：正在使用（正在侦听）的TCP套接字数量。其值≤ netstat –lnt | grep ^tcp | wc –l
TCP: orphan：无主（不属于任何进程）的TCP连接数（无用、待销毁的TCP socket数）
TCP: tw：等待关闭的TCP连接数。其值等于netstat –ant | grep TIME_WAIT | wc –l
TCP：alloc(allocated)：已分配（已建立、已申请到sk_buff）的TCP套接字数量。其值等于netstat –ant | grep ^tcp | wc –l
TCP：mem：套接字缓冲区使用量（单位不详。用scp实测，速度在4803.9kB/s时：其值=11，netstat –ant 中相应的22端口的Recv-Q＝0，Send-Q≈400）
UDP：inuse：正在使用的UDP套接字数量
RAW：
FRAG：使用的IP段数量
IPv6请看：cat /proc/net/sockstat6
TCP6: inuse 3 UDP6: inuse 0 RAW6: inuse 0 FRAG6: inuse 0 memory 0
通过这些值，可以很容易计算出当前的tcp请求数，然后做相关的监控。
puppet终于启用了，先在小范围使用，熟练了再推广~
HandlerSocket的php扩展已出，太棒了，一会就测试下~
测试结果呢？
还没测试，sorry
多多交流，这是我安装的记录，周边的扩展还不是很完备http://blog.sina.com.cn/s/blog_550ffb0b0100nowb.html
儿童教育软件大全
上一周在微博架构与平台安全演讲中提到多IDC及架构设计的方法，由于最近工作中经常碰到这种情况，再举一个小案例补充一下。
Web数据访问比较好的设计模式是使用cursor方式（参考前文用Twitter的cursor方式进行Web数据分页），原理上相当于增量方式访问数据，可以极大提高访问性能。
在单IDC场景中，如图1，系统的id是递增，假设用户上一次访问最新一条记录是1002，则本次访问最佳的方式是 get?cursor=1002，可以高效取到后面3条新记录。
多IDC场景，看图2，假设白色背景属于Region 1，灰色背景属于Region 2, 由于两地同步有延迟，这样在Region 1中1001和1003来到时间较晚，排在本地数据1002和1004后面。假设用户上一次也是取到最新一条是1002(注意此时1001没取到，因为从外地未同步过来)。在Region 1调用 get?cursor=1002返回结果会得到什么？从数据库角度来看，访问cursor=1002 只会取到id>1002的记录，而上次未取到的1001即使已经同步过来是永远不会返回了。这样就产生了数据一致性问题，1001丢了。另外一个机房Region 2调用也产生类似问题。不同的cursor产生不同的丢失问题。
提出这个问题后身边很多技术人员非常感兴趣，经常走在路上被拦住介绍他们突然想到的一种更巧妙的解决方法。部分思路如下
(这里先不考虑ID递增算法如何实现，多IDC使用K-SORT方式递增也是比较容易的)
大部分方法貌似都能工作，但都有问题或不完美，更重要的一点，也就是上周演讲中提到的，架构要把复杂的问题抽象简单，很多技术人员面对这个问题，并没有深层次思考这个场景的问题本质是什么，因此虽然匆匆考虑了很多复杂的解决方案，但是没有完美解决问题。
有兴趣的朋友可以继续思考，看能否将复杂的问题抽象简单并解决？
在这样的环境中，绝大部分是php首先出现瓶颈，如果没有必要的监控就抓瞎了，下面是我们现在的做法：
1、在nginx里增加以下日志的记录
$request_time: nginx处理请求的时间
$upstream_response_time: php-cgi的响应时间
2、php-fpm加上慢请求的日志
3、使用xhprof来精准定位
其实通过1、2的日志，基本上可以确认问题所在了，使用xhprof当然就更完美了，在实际生产环境中，加上xhprof，貌似都是比较事后了（大部分情况下xhprof都不默认启动）。
PHP-DWS
1. 关于 php-dws
php-dws 是 PHP Direct Web Server 的缩写, 是针对 php 的一个新型 sapi 工作模块,
通过 dwsgi 协议与 web server (如 nginx) 协同工作.
之所以叫 Direct , 是因为它在执行 php 脚本过程中是直接把输出结果传递给 http client 的
而不是转交 webserver(nginx) 再由 webserver 发送给 http client.
在多数情况下, php-dws 可以用于取代 php/fastcgi 的工作, 并且能更出色的完成.
2. 安装与配置
相对比较复杂，包括 NGINX 模块和 PHP的 sapi 模块2个部分，详细进入我的论坛查看吧
http://www.hightman.cn/bbs/forumdisplay.php?fid=15
3. 后话
php-dws 还是一个实验性的产品，不推荐运行在生产环境，由于其另类的方式在某些WEB应用的
场合可能有需求。php-dws 的特别应用示例
或点击查看这个刷新显示效果：http://root.twomice.net/dws-test/sleep.phpd
From： http://www.hightman.cn/bbs/forumdisplay.php?fid=15
关于这个设计思路，感觉挺有意思，作者加油，关注下:)
升级了运维架构师交流QQ群：35844310，欢迎您的加入^_^
一般认为NoSQL数据库在性能方面要优于传统的SQL数据库。但是有两个SQL的解决方案宣布：对于大型系统的高可扩展性需求，SQL仍然是可行的解决方案！这两个SQL解决方案分别是MySQL加NoSQL层插件和支持SQL的VoltDB数据库。
Yoshinori Matsunobu是Sun/Oracle的前雇员，从事MySQL的研发工作，目前是DeNA的首席数据库和基础设施架构师，他以插件的方式为MySQL/InnoDB提供解决方案，可以在一台2.53GHZ、8核CPU、32G内存的Nehalem服务器上把每秒的查询数量（qps）提升到750,000以上。在同样的硬件环境下，无插件的MySQL只能提供100,000左右的qps，如果使用memecached的话，可以增加到大约400,000。经过对RDBMS的分析，Matsunobu意识到大部分时间都花在SQL的开销上，比如调用MYSQLparse()、MYSQLlex()、make_join_statistics()和JOIN::optimize()等。他写到：
很显然性能降低的原因主要在SQL层，而不是“InnoDB（存储）”层。MySQL必须做很多事情……但memcached/NoSQL是不需要做这些额外工作的。
SQL层的功能包括解析SQL语句、打开/锁定/解锁/关闭表、解决并发问题等。Matsunobu的解决方案就是增加额外的NoSQL层：
我们认为最好的方式就是在MySQL内部实现一个NoSQL的网络服务器。也就是说，编写一个网络服务器作为MySQL的插件（守护插件），用来监听特定端口，接收NoSQL的协议和API，然后通过MySQL内部存储引擎API直接访问InnoDB。这种方式很像NDBAPI，不同的是它可以与InnoDB交互。
他的团队开发了HandlerSocket插件，有了这个插件，MySQL更像一个NoSQL数据库，通过监听一个独立的端口，接收从SQL层来的简单查询请求，例如主键查询，索引扫描和插入/更新/删除。这一变化把数据库性能提升到了750K qps以上。常用端口可以接收处理复杂查询，其核心仍然是SQL数据库。DeNA采用SQL/NoSQL混合的方式取得了成功，据Matsunobu所言，在相同的时间内，这种解决方案把多个memcached和MySQL主从服务器的方案远远甩在了后面。
另一个很有希望的SQL解决方案是VoltDB，这是一个内存中的开源OLTP SQL数据库，能够保证事务的完整性（ACID）。VoltDB是由原Ingres和Postgres的架构师Mike Stonebraker设计的。该数据库主要特征如下：
- 为了获得最大化吞吐量，数据保存在内存中（而不是在硬盘），这样可以有效消除缓冲区管理。
- VoltDB通过SQL引擎把数据分发给集群服务器的每个CPU进行处理。
- 每个单线程分区自主执行，消除锁定和闩锁的需求。
- VoltDB可以通过简单的在集群中增加附加节点的方式实现性能的线性增加。
正如其开发者宣称的那样，该数据库的性能使其成为NoSQL解决方案的有力竞争者：
- VoltDB在单节点上可以每秒处理53000个事务请求（TPS），其他DBMS在相同的硬件环境下只能处理1155个。VoltDB的扩展是近似线性的──在12个节点的VoltDB集群上进行同样测试，可以处理560,000 TPS。
- 基准案例：某个客户的在线游戏在12个节点的VoltDB集群上处理了130万 TPS。
- VoltDB还针对NoSQL的键-值存储方式作了基准测试，VoltDB在处理各种键-值存储负载的情况下获得了相同或更好的性能。
除了它的性能，VoltDB的主要优势是可以与SQL用户进行交流，这些SQL用户是很好的资源。
近期还会推出VoltDB的企业版本，包括基于浏览器的数据库管理系统，提供、管理和监控数据库集群。除了免费的社区版本，针对企业版的支持也开始了。
查看英文原文：MySQL/HandlerSocket and VoltDB: Contenders to NoSQL
From: http://hi.baidu.com/%CA%AB%D5%B9/blog/item/30e9aed38d3af024960a1649.html
安装beanstalkd（队列）
官网：http://kr.github.com/beanstalkd/
注意：需要先安装libevent，另外可以在：http://wiki.github.com/kr/beanstalkd/client-libraries 这里找到php客户端，目前只有php class。
1、下载
wget -c http://xph.us/dist/beanstalkd/beanstalkd-1.4.6.tar.gz
2、编译
tar zxvfp beanstalkd-1.4.6.tar.gz cd beanstalkd-1.4.6 ./configure --prefix=/opt/app/beanstalkd --with-event=/opt/app/libevent/ make && make install
3、启动
/opt/app/beanstalkd/bin/beanstalkd -d -b /opt/app/beanstalkd/db/ -f 60000 -l 192.168.4.75 -p 11300 -u nobody
和memcached非常类似
具体参数如下：
Options: -d detach -b DIR binlog directory (must be absolute path if used with -d) -f MS fsync at most once every MS milliseconds (use -f 0 for "always fsync") -F never fsync (default) -l ADDR listen on address (default is 0.0.0.0) -p PORT listen on port (default is 11300) -u USER become user and group -z BYTES set the maximum job size in bytes (default is 65535) -s BYTES set the size of each binlog file (default is 10485760) -v show version information -h show this help
4、相关文档
http://wiki.github.com/kr/beanstalkd/faq
http://adam.heroku.com/past/2010/4/24/beanstalk_a_simple_and_fast_queueing_backend/
运维工作：配置自动化，监控智能化~
amazon的EC2确实很方便，但是计费方式还不够灵活~
silentime, 
peter are discussing.
soz 09:02 on 2010 年 11 月 25 日 链接地址
现在总是发微薄啊。。。
深夜的蚊子 09:25 on 2010 年 11 月 25 日 链接地址
给点使用思路，我最近也在思考这个东西怎么用
bixuan 09:37 on 2010 年 11 月 25 日 链接地址
呵呵，微博方便啊:)
bixuan 09:39 on 2010 年 11 月 25 日 链接地址
没啥特别好的思路。加油吧