Tuesday, September 18, 2007

Uploading Large Files With PHP

Let me first start by saying that I’m currently working at a startup, and for anyone who’s ever done this, you know it sucks up a LOT of your time… in my case pretty much all of it. I’m not complaining by any means, I’m doing some of the coolest work I’ve ever done in fact! However, I just wanted to explain my little posting hiatus…

Anyway, on to the matter at hand. I have found myself in a position where I am writing scripts that may need to upload fairly large files. My scripts were timing out, and I couldn’t seem to figure out why. For the unitiated, there are some standard things that you usually do to both your php.ini and in your script in this situation:

  • Set your max_upload_size in php.ini higher (it’s only 2M out of the box)
  • Set your script timeout to never (set_time_limit(0); in your scripts… don’t do this in your php.ini)

However, it turns out there are some other php.ini config variables that you may need to look at:

  • memory_limit - This may also be an obvious one to some people, but for those of you who don’t know, this restricts how much memory PHP is allowed to consume while processing. When working with images and large files, this needs to be upped to accommodate these needs.
  • post_max_size - This was the one killing me, and it was a major “Duh” moment as well. If you aren’t allowed to create a large post, how can you expect to upload a large file?? This should be set the same as your max_upload_size.
  • max_input_time - the time that the script should spend in accepting input. This is setting defaults to 60 seconds, and you will probably need to update this as well

That’s about it! Quick and simple solution to a problem that is probably pretty common. I promise my next post won’t disappoint (and should happen soonish)… I’ll be talking about how to set up a MySQL replication environment on one server!

Monday, March 05, 2007

2.6.9-42.EL不編譯kernel給iptables添加模塊(IPP2P、L7-filter)

注意:經過本人的實驗,用SMP加載L7-filter時,只要REBOOT或者長時間開啓服務器,是
會導致Kernel Crashes的,故請在加載L7-filter前,轉用SMP以外的Kernel.

以下是L7-filter官方網站上的一句話:
“Some users have reported kernel crashes when they using SMP with l7-filter. (Some have also reported that their SMP systems run fine.) If you have a multi-CPU machine, test carefully before putting it into production with l7-filter.”

安裝環境:Redhat ES 4 Update4
kernel 2.6.9.42.EL

需要準備以下包:
l7-protocols-****.tar.gz (可以在 http://l7-filter.sourceforge.net/下載,用於定義L7可用的
Protocols);
netfilter-layer7-VERSION.tar.gz (可以在http://l7-filter.sourceforge.net/下載,L7的可編譯
代碼)
patch-o-matic-ng-20060706.tar.bz2 (可以在http://www.netfilter.org/下載,最後一個版本
的ipp2p模塊的tar包)
iptables-VERSION.tar.bz2 (可以在http://www.netfilter.org/下載,最新版本的iptables)

kernel-2.6.9-42.EL.src.rpm (可以在Redhat source code dick中尋找)
(注:因為Redhat es4不在包括source code,故/usr/src/下沒有所需的kernel source)

獲得kernel source:
kernel-2.6.9-42.EL.src.rpm放在/home/下;
# rpm -ivh kernel-2.6.9-42.EL.src.rpm
# cd /usr/src/redhat/SPECS
# rpmbuild -bp --target=i686 kernel-2.6.9.spec
# cp -a /usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9 /usr/src/linux/

初始化Kernel:
# cd /usr/src/linux/
# vi Makefile
(將改EXTRAVERSION = -42.EL)
# make mrproper
# make clean
# make menuconfig
(什麽也不做,直接save跳出來)
為了生成.config文件

IPTABLES:
將iptables下載到/home
# tar -xvjf /home/iptables-VERSION.tar.bz2 /usr/src/
# mv iptables-VERSION iptables

添加ipp2p模塊到Kernel
下載ipp2p到/home下
# tar -xvjf /home/patch-o-matic-ng-20060706.tar.bz2 /usr/src/
# cd patch-o-matic-ng-20060706/
# ./runme ipp2p
當出現KERNEL_DIR=/usr/src/linux 和 IPTABLES_DIR=/usr/src/iptables直接回車就可以了,

添加L7-filter:
下載
netfilter-layer7-VERSION.tar.gz 到/home下
# tar -xvzf /home/netfilter-layer7-VERSION.tar.gz /usr/src/
# mv netfilter-layer7-VERSION netfilter-layer7
# cd /usr/src/linux
# patch -p1 <../netfilter-layer7/for_older_kernels/2.6.9****.patch
# cd /usr/src/iptables
# patch -p1 < ../netfilter-layer7/iptables-layer7-***.patch
# cd /usr/src/iptables/extensions/
# chmod +x .layer7-test

添加Kernel的模塊:
# cd /usr/src/linux
# make menuconfig
Device Drivers->;Networking support->;Networking options->;Network packet filtering (replaces ipchains)->;IP: Netfilter Configuration將IPP2P和Layer7選上;
保存后跳出

安裝模塊:
# cd /usr/src/linux
# make modules
當編譯完HOSTLD scripts/modpost后用ctrl+c中止;
# make modules SUBDIRS=net/ipv4/netfilter
這裡生成了2.6kernel才會有的.o文件
# chmod +x /usr/src/linux/net/ipv4/netfilter/*.ko
# cp /usr/src/linux/net/ipv4/netfilter/*.ko /lib/modules/2.6.9-34.EL/kernel/net/ipv4/netfilter
# depmod -a

編譯iptables:
# vi /usr/src/linux/include/linux/config.h
註釋掉:
//#ifndef __KERNEL__
//#error including kernel header in userspace; use the glibc headers instead!
//#endif
# cd /usr/src/iptables
# make KERNEL_DIR=/usr/src/linux BINDIR=/sbin LIBDIR=/lib MANDIR=/usr/share/man
# make KERNEL_DIR=/usr/src/linux BINDIR=/sbin LIBDIR=/lib MANDIR=/usr/share/man install

將ipp2p、l7-filter加入:
l7-protocols-****.tar.gz下載到/home下
# tar -xvzf l7-protocols-****.tar.gz
# make install
# modprobe ipt_ipp2p
# modprobe ipt_layer7
# lsmode
Module Size Used by
ipt_limit 2816 3
ipt_ipp2p 6912 2
ipt_layer7 10412 4
ipt_LOG 6144 12
ipt_state 2176 12
iptable_nat 21948 1

測試iptables:
# iptables -A FORWARD -p ALL -m layer7 --l7proto bittorrent -j LOG --log-level DEBUG --log-prefix "bittorrent Outgoing Log:"
# iptables -A FORWARD -p ALL -m ipp2p --edk --kazaa --gnu --bit --apple --dc --soul --winmx --ares -j LOG --log-level DEBUG --log-prefix "IPP2P Outgoing Log:"
# iptables -vnL