Monthly Archives: March 2013

为Apache rewrite转发添加POST数据支持

apache的rewrite转发能够处理大部分的,例如在wordpress上的URL虚拟静态化:
apache_rewrite_POST_1

但是,如果rewrite的url需要处理POST数据的话,这样的转发就不行了,因为apache在转发的时候是默认没有把POST数据也交给rewrite的对象的,类似这样:
apache_rewrite_POST_2

需要在rewriterule后加上参数[P].P表明force proxy,需要mod_proxy以及根据相应协议的proxy模块比如mod_http_proxy
实际上这个时候apache就充当了反向代理的功能。

RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteCond %{REQUEST_URI} ^/user\.php$
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,P]

很遗憾,我第一次这样做的时候,apache给了我一个”500 Internal Server Error”,error log报错如下

[Thu Mar 07 21:36:07 2013] [error] [client 63.223.121.79] SSL Proxy requested for 63.223.121.79:80 but not enabled [Hint: SSLProxyEngine]
[Thu Mar 07 21:36:07 2013] [error] proxy: HTTPS: failed to enable ssl support for 63.223.121.79:443 (www.test.com)

但是如果简单的把一个http uri转发至另一个http uri,用P参数就OK了,很正常.

显然,ssl的转发会更加复杂一些,你看已经涉及SSLProxyEngine.没有猜错的话,apache好像是这样处理问题的:
apache_rewrite_POST_3

毕竟http进程和https分别在80和443端口监听,如果rewrite http至https,其中甚至需要把Apache当做Proxy?!

Python异步网络探索(一)

异步网络据说能极大的提高网络server的连接速度,所以打算写一个专题,来学习和了解异步网络.因为Python有个非常出名的异步Lib:Twisted,所以就用Python来完成.

OK,首先写一个pythone socket的server段,对开放三个端口:10000,10001,10002.krondo的例子中是每个server绑定一个端口,测试的时候需要分别开3个shell,分别运行.这太麻烦了,就分别用三个Thread来运行这些services.

import optparse
import os
import socket
import time
from threading import Thread
import StringIO

txt = '''1111
2222
3333
4444
'''

def server(listen_socket):
    while True:
        buf = StringIO.StringIO(txt)
        sock, addr = listen_socket.accept()
        print 'Somebody at %s wants poetry!' % (addr,)
        while True:
                try:
                    line = buf.readline().strip()
                    if not line:
                        sock.close()
                        break
                    sock.sendall(line)  # this is a blocking call
                    print 'send bytes to client:%s' % line
                    #sock.close()
                except socket.error:
                    sock.close()
                    break
                time.sleep(1)  #server和client连接后,server会故意每发送一个单词后等待一秒钟后再发送另一个单词


def main():
    ports = [10000, 10001, 10002]
    for port in ports:
        listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        addres = (str('127.0.0.1'), port)
        listen_socket.bind(addres)
        listen_socket.listen(5)
        print "start listen at:%s" % (port,)
        worker = Thread(target = server, args = [listen_socket])
        worker.setDaemon(True)
        worker.start()


if __name__ == '__main__':
    main()
    while True:
        time.sleep(0.1) #如果不sleep的话,CPU会被Python完全占用了
        pass

下面是一个client,没有才用异步网络,连接这个三个端口的server:

import socket


if __name__ == '__main__':
    ports = [10000, 10001, 10002]
    for port in ports:
        address = (str('127.0.0.1'), port)
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock.connect(address)
        poem = ''
        while True:
            data = sock.recv(4)
            if not data:
                sock.close()
                break
            poem += data
        print poem

Server端运行后:
屏幕快照 2013-03-17 下午11.15.26

在server.py中,故意在每次发送单词后server段等待1秒钟(代表现实情况中可能发生的各种延迟,例如数据库读取、硬盘IO等),这样的话,每个socket连接都需要4秒钟,分别读取三次就需要12秒钟。

sync

但若是在异步socket的情况下,工作的流程就是这样:

async

下面用异步的client来读取,代码如下:

import datetime, errno, optparse, select, socket

def connect(port):
    """Connect to the given server and return a non-blocking socket."""
    address = (str('127.0.0.1'), port)
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect(address)
    sock.setblocking(0)
    return sock

def format_address(address):
    host, port = address
    return '%s:%s' % (host or '127.0.0.1', port)

if __name__ == '__main__':
    ports = [10000, 10001, 10002]
    start = datetime.datetime.now()

    sockets = map(connect, ports)
    poems = dict.fromkeys(sockets, '') # socket -> accumulated poem 

    # socket -> task numbers
    sock2task = dict([(s, i + 1) for i, s in enumerate(sockets)])
    sockets = list(sockets) # make a copy

    while sockets:
        #运用select来确保那些可读取的异步socket可以立即开始读取IO
        #OS不停的搜索目前可以read的socket,有的话就返回rlist
        rlist, _, _ = select.select(sockets, [], [])
        for sock in rlist:
            data = ''
            while True:
                try:
                    new_data = sock.recv(1024)
                except socket.error, e:
                    if e.args[0] == errno.EWOULDBLOCK:
                        break
                    raise
                else:
                    if not new_data:
                        break
                    else:
                        print new_data
                        data += new_data

            task_num = sock2task[sock]
            if not data:
                sockets.remove(sock)
                sock.close()
                print 'Task %d finished' % task_num
            else:
                addr_fmt = format_address(sock.getpeername())
                msg = 'Task %d: got %d bytes of poetry from %s'
                print  msg % (task_num, len(data), addr_fmt)

            poems[sock] += data

    elapsed = datetime.datetime.now() - start
    print 'Got poems in %s' %  elapsed

结果只需要4秒就完成了读取任务。效率是刚才同步socket的三倍。对客户端的异步改造主要有两点:

  • 同步模式下,客户端分别创建socket;而在异步模式下,client开始就创建了所有的socket。
  • 通过“sock.setblocking(0)”设置socket为异步模式。
  • 通过Unix系统的select俩返回可读取IO
  • 最为核心的是26行和29行。尤其是29行的select操作返回待读取socket的列表。

参考:
1. An Introduction to Asynchronous Programming and Twisted
2. Python 2.7 Socket Document
3. Python 2.7 select

Mysql数据库仅运行本机连接

为了安全性的原因,可能需要把Mysql设置为近可以通过127.0.0.1才能连接.

1. 首先更改my.cnf,绑定127.0.0.1

bind-address  = 127.0.0.1

这样只能本机访问,如果改成外网地址则外网可以访问mysql端口.

2. 添加iptables防火墙,阻止外部对3306端口的连接

iptables -A INPUT -i eth0 -p tcp -m tcp --dport 3306 -j DROP

如果想要开启的话:

iptables -A INPUT -i eth0 -p tcp -m tcp --dport 3306 -j ACCEPT