Category: Linux

why i receive zero in my socket

I have two programs, one written in c++, the other is in java. C++ keep sending bytes to java, overtime write out 400KB. Java keep receiving the data using the readBytes function, this function returns an integer which is the number of byte it actually read for the function call. For every send (400kb) in c++ and every read in java, I print out something, so I know c++ is 2 times faster than Java. That mean c++ is sending out the bytes in 2-times speed than java. After 10 seconds, on java side I read out zero from the byte array. But the return value of readBytes() function is still a positive number. Why behave like this? Java should return -1 if no packet or packet lost.

To fix this problem, for every 400kb read from java, I write out the string “end” to c++, c++ have to receive that “end” string before send out another 400kb. So c++ and java are doing handshaking, they align they send speed and write speed, preventing the c++ is sending faster than java. Now, I get all the bytes correctly. So in http, server side and client side *MUST* do handshake. (Correct me if I am wrong)

If the wget is doing handshake to apache server, even in a very slow network, we are very hard to have packet lost. Unless the bandwidth still can’t send out the bytes array in ONE single send() function. But in real case, we usually split the data into very small piece, let say 4KB. So we are very very hard to see packet lost. This explains why I can’t see packet lost/ECN, even I use wget to use up the whole bandwidth.

read count : 223

[solved] usb wireless mouse freeze periodically in ubuntu 12.04 lts

usb wireless mouse freeze periodically in ubuntu 12.04 lts, edit /boot/grub/grub.cfg, add a boot option irqpoll will fix it

menuentry 'Ubuntu, with Linux 3.8.0-41-generic' --class ubuntu --class gnu-linux --class gnu --class os {
	recordfail
	gfxmode $linux_gfx_mode
	insmod gzio
	insmod part_msdos
	insmod ext2
	set root='(hd0,msdos1)'
	search --no-floppy --fs-uuid --set=root 4abd314f-d625-417d-b9f1-52ae7df8d1fd
	linux	/boot/vmlinuz-3.8.0-41-generic root=UUID=4abd314f-d625-417d-b9f1-52ae7df8d1fd ro   quiet splash nomodeset $vt_handoff irqpoll
	initrd	/boot/initrd.img-3.8.0-41-generic
}

read count : 940

We should mix 4K and 4M Page

I was fixing the PSE and PAE page table display for GKD, I found out Linux using 4M page to map large address space and it uses 4K page to map broken space. So advance OS should mix use of 4K and 4M page.

how linux use 4k and 4m page together

 

Now GKD supports PSE=1 (4M page)

4M page

read count : 596

I think i am under dns attack

I think i am under dns attack, in my /var/log/message i keep seeing these. Someone is using over 2000 ip to lookup my dns server.

Jul  8 07:14:39 petersoft named[3263]: client 65.254.249.169#31655: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 32.44.116.12#45668: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 23.247.81.239#32733: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 36.234.182.148#65128: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 32.253.30.9#24067: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 83.10.132.93#10355: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 124.12.88.39#20033: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 16.251.239.11#3843: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 15.105.18.26#12012: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 35.145.149.224#56066: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 72.250.71.141#55169: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 16.40.135.80#42400: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 108.147.217.226#45322: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 8.108.103.84#30252: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 91.206.27.44#53448: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 94.243.234.26#42498: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 120.167.243.44#57980: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 115.168.23.42#64634: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 14.85.197.38#57296: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 8.198.131.140#37656: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 1.74.160.47#41155: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 106.144.248.143#52145: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 101.189.35.99#60637: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 121.251.93.58#20274: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 24.104.44.88#23336: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 119.150.51.199#8437: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 92.12.35.97#55673: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 66.70.220.236#24442: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 18.247.171.123#53097: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 107.205.171.181#33105: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 78.48.204.24#26234: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 73.52.107.198#64558: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 15.69.210.99#61165: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 39.47.93.102#43460: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 5.65.55.240#16498: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 68.18.205.222#21508: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 128.236.221.133#56671: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 77.72.147.216#11370: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 49.212.253.82#24316: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 66.254.251.100#32610: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 91.74.37.67#55767: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 112.102.110.83#19745: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 85.165.163.23#19555: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 14.136.98.212#32228: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 67.97.28.244#41398: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 60.217.137.235#415: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 41.120.87.227#43219: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 31.28.113.137#44481: no more recursive clients: quota reached
Jul  8 07:14:39 petersoft named[3263]: client 105.44.182.212#34606: no more recursive clients: quota reached

read count : 387

My new server

Just bought a 2nd hand HP server (8core + 16GB ram), cost me $2630 HKD. One 1TB 2.5″ hard disk cost me $450. Total $3080. It reboot only take 20 seconds, dell needs 5 mins (CRAZY!!!)

 

/root>cat /proc/cpuinfo 
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz
stepping	: 10
microcode	: 0xa07
cpu MHz		: 2500.059
cache size	: 6144 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 xsave lahf_lm dtherm tpr_shadow vnmi flexpriority
bogomips	: 5000.11
clflush size	: 64
cache_alignment	: 64
address sizes	: 38 bits physical, 48 bits virtual
power management:

2nd hand server

read count : 616

Analyst android ELF dependency-graph by elf-dependency-walker

Someone asked me in email about using elf-dependency-walker to analyst library or executable in linux. I found out android is using ELF format for its libraries or executables too. So the rest is simple. Please download the android emulator, within the directory there is the system image, which contain the file system.

system image
system image

You can see in the above graph, it is a file called “system.img”. It is the file system image in ext4 format. So you can mount it by this command:

mkdir temp
mount -o loop system.img temp
mount it
mount it

Now, cd to the temp directory, you can see all android files in there.

change the lookup directory path in elf-dependency-wwalker
change the lookup directory path in elf-dependency-wwalker

Just start elf-dependency-walker by “java -jar elf-dependency-walker.jar”. You can download it in https://code.google.com/p/elf-dependency-walker. Click on “Setting”, and place the full path of your mounted directory (The directory you mounted to your android fs) in it.

select any executable/library from the android fs
select any executable/library from the android fs

Click “Analyst” button, choose any file (must be elf format file).

eld chart is generated
eld chart is generated

It will analyst the dependency for you.

read count : 731

solved : compile error in ffmpeg “error: invalid combination of opcode and operands”‏

I am using ubuntu 12.04 64 bits and trying compile ffmpeg and got this error:

libavfilter/x86/vf_pullup.asm:59: error: invalid combination of opcode and operands
libavfilter/x86/vf_pullup.asm:138: error: invalid combination of opcode and operands
libavfilter/x86/vf_pullup.asm:175: error: invalid combination of opcode and operands

To solve this, add “–disable-mmx” to your ./configure

read count : 1056

sox and ffmpeg (convert hls into live hls) usage

These are the basic usage of SOX

# Create background noise profile from mp3
/usr/bin/sox noise.mp3 -n noiseprof noise.prof
 
# Remove noise from mp3 using profile
/usr/bin/sox input.mp3 output.mp3 noisered noise.prof 0.21
 
# Remove silence from mp3
/usr/bin/sox input.mp3 output.mp3 silence -l 1 0.3 5% -1 2.0 5%
 
# Remove noise and silence in a single command
/usr/bin/sox input.mp3 output.mp3 noisered noise.prof 0.21 silence -l 1 0.3 5% -1 2.0 5%
 
# Batch process files
/usr/bin/find . -type f -name "*.mp3" -mmin +30 -exec sox -S --multi-threaded -buffer 131072 {} /path/to/output/{} noisered noise.prof 0.21 silence -l 1 0.3 5% -1 2.0 5% \;
 
# Remove insignificant files
/usr/bin/find . -type f -name "*.mp3" -mmin +30 -size -500k -delete

This script will convert ffmpeg HLS into “live streaming HLS”

#!/bin/bash

x=0
while true; do
	#cat out.m3u8|sed 's/YES/NO/' | sed 's/#EXT-X-ENDLIST//'> temp
	line=`wc -l out.m3u8|awk '{print $1}'`
	line=$(($line-5))
	line=$(($line/2))
	echo $line

	head -5 out.m3u8 |sed 's/YES/NO/' | sed 's/#EXT-X-ENDLIST//' | sed "s/#EXT-X-MEDIA-SEQUENCE:0/#EXT-X-MEDIA-SEQUENCE:$line/" > temp
	#echo "#EXTM3U">temp
	#echo "#EXT-X-VERSION:3">>temp;
	#echo "#EXT-X-TARGETDURATION:2">>temp;
	#echo "#EXT-X-MEDIA-SEQUENCE:$x">>temp;
	tail -16 out.m3u8 >> temp;
	#echo "#EXTINF:1,">>temp;
	#echo "dummy.mp3">>temp;

	tail -8 out.m3u8 | grep mp3 | while read a; do
		sox $a c$a noisered noise.prof 0.21
	done	

	sed -i 's/a0/ca0/' temp

	mv temp out2.m3u8
	x=$(($x+1))
	#sleep 2
done

The above script has to run with ffmpeg:
/root/download/ffmpeg-2.1.4/ffmpeg -f alsa -ac 2 -i hw:0,0 -strict experimental -acodec libmp3lame -map 0 -f segment -segment_list out.m3u8 -segment_format libmp3lame -segment_time 0.5 -segment_list_flags +live ‘a%05d.mp3’

read count : 678