Meet @fannheyward

Linux Performance Analysis in 60s

uptime/w --------------------> load average
dmesg | tail ----------------> kernel errors
vmstat 1 --------------------> overall stats every second
mpstat -P ALL 1 -------------> CPU balance
pidstat 1 -------------------> process usage, every second
iostat -xz 1 ----------------> disk I/O
free -m ---------------------> memory usage
sar -n DEV 1 ----------------> network I/O
sar -n TCP,ETCP 1 -----------> TCP stats
top -------------------------> check overview

dmidecode

You can use dmidecode to display server physical info, for example RAM max capacity.

dmidecode -t 16

Handle 0x0032, DMI type 16, 15 bytes
Physical Memory Array
	Location: System Board Or Motherboard
	Use: System Memory
	Error Correction Type: Multi-bit ECC
	Maximum Capacity: 48 GB
	Error Information Handle: Not Provided
	Number Of Devices: 6

App Store Front Code

X-Apple-Store-Front header is needed to scrape in App Store.

// 29 or 26 or 9
CN 143465-19,29
US 143441-1,29
JP 143462-9,29
KR 143466-13,29
HK 143463-18,29
AU 143460,29
TW 143470-18,29
CA 143455-6,29
DK 143458-2,29
RU 143469-16,29
ID 143476-2,29
TR 143480-2,29
GR 143448-2,29
DE 143443-4,29
IT 143450-7,29
NO 143457-2,29
FR 143442-3,29
TH 143475-2,29
SE 143456-17,29
FI 143447-2,29
GB 143444,29
NL 143452-10,29
BR 143503-15,29
PT 143453-24,29
MX 143468-28,29
ES 143454-8,29
VN 143471-2,29

App info:

curl -H 'x-apple-store-front: 143465-19,29' 'https://itunes.apple.com/cn/app/id1318151064?mt=8'

Top Free iPhone Apps:

curl -H 'x-apple-store-front: 143465-0,9'   'https://itunes.apple.com/WebObjects/MZStore.woa/wa/topChartFragmentData?popId=27&pageNumbers=0&pageSize=1000'
curl -H 'x-apple-store-front: 143465-19,29' 'https://itunes.apple.com/WebObjects/MZStore.woa/wa/viewTop?id=25204&popId=27&genreId=36'

Rating & reviews:

curl -H 'x-apple-store-front: 143441-0,9'  'https://itunes.apple.com/us/customer-reviews/id284882215?dataOnly=true&displayable-kind=11'
curl -H 'x-apple-store-front: 143441-1,29' 'https://itunes.apple.com/us/customer-reviews/id284882215?dataOnly=true&displayable-kind=11'
curl -H 'x-apple-store-front: 143441-1,29' 'https://itunes.apple.com/WebObjects/MZStore.woa/wa/userReviewsRow?id=284882215&displayable-kind=11&startIndex=0&endIndex=100&sort=1'

Search:

curl -H 'x-apple-store-front: 143441-1,29' 'https://itunes.apple.com/WebObjects/MZStore.woa/wa/search?clientApplication=Software&term=NBA'
curl -H 'x-apple-store-front: 143441-1,29' 'https://search.itunes.apple.com/WebObjects/MZSearchHints.woa/wa/trends?maxCount=10'
curl -H 'x-apple-store-front: 143465-1,29' 'https://search.itunes.apple.com/WebObjects/MZSearchHints.woa/wa/hints?clientApplication=Software&term=NBA'

Nginx map

map $room $room_server {
    default 192.168.1.101:8080;
    1 192.168.1.101:8080;
    2 192.168.1.102:8080;
}

server {
    listen  80;

    location ~ /api/(\d+)/room {
        set $room $1;
        echo $room_server;
    }
}

split_clients 类似的结构，可以用来做请求 A/B 测试:

split_clients $arg_app_key $variant {
    0.5%    .one;
    2.0%    .two;
    *       "";
}

nginx mirroring tips and tricks

Shell Notes

&& || ; in shell command:

  • cmd1 && cmd2 means cmd2 will only run while cmd1 success.
  • cmd1 || cmd2 means cmd2 will only run while fmd1 fails.
  • cmd1 ; cmd2 will run cmd2 regardless cmd1 success or not.

通过 Debian Snapshot 安装旧版本包

某个项目需要 PHP 5.3 支持，通过 APT 没办法直接安装，编译安装又是一大堆依赖，最后通过 Debian Snapshot 解决。

  1. 在 http://snapshot.debian.org/ 搜索需要的包, 比如 php5
  2. Got http://snapshot.debian.org/archive/debian-ports/20120225T023111Z/pool-m68k/main/p/php5/
  3. 添加到 source.list:
deb http://snapshot.debian.org/archive/debian/20120225T023111Z/ unstable main
deb-src http://snapshot.debian.org/archive/debian/20120225T023111Z/ unstable main
  1. apt-get -o Acquire::Check-Valid-Until=false update
  2. apt-get install php5=5.3.10-2 php5-fpm php5-cgi, done.

docker-compose for MySQL + phpMyAdmin

version: '3'
services:
  mysql:
    image: 'mysql:5'
    container_name: 'mysql'
    command: mysqld --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci --init-connect='SET NAMES UTF8;'
    ports:
      - "3306:3306"
    environment:
      MYSQL_ROOT_PASSWORD: 'test'
  phpmyadmin:
    image: 'phpmyadmin/phpmyadmin'
    container_name: 'phpmyadmin'
    links:
      - mysql
    ports:
      - '8080:80'
    environment:
      PMA_HOST: mysql

Logging mode: Push vs. Pull

日志收集或监控系统有两种工作模式，Push or Pull：

  • Push 是在应用内自己主动把监控指标推送到监控系统
  • Pull 是应用把指标按照指定的格式暴露出来，由监控系统定期的抓取收集，看起来像是监控系统从应用拉取

Push mode:

  • 对于 event-drive 类型的监控更为灵活方便
  • 应用多点部署不影响监控
  • 可能对监控系统 DDos
  • 要注意推送频率控制，以及失败后的重试机制

Pull mode:

  • 方便控制收集频率，对应用压力可控，侵入也小
  • 更及时的发现服务宕机
  • 需要大量配置监控接入点，尤其是应用集群化
  • 指标不够实时，顺序可能错乱
  • 监控指标一般是通过 log 文件或 HTTP 接口对外暴露，parser 较为复杂

Push:

Pull:

macOS Python env

brew install python

which python  // /usr/bin/python
which python3 // /usr/local/bin/python3

sudo easy_install neovim
pip3 install --upgrade neovim

多版本共存还可以用 pyenv 解决。

使用 pipsi 安装 Python-base 工具，比如 ansible，httpie，pylint，yapf 等：

sudo /usr/bin/easy_install virtualenv
curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | /usr/bin/python

pipsi install 'python-language-server[all]'
pipsi install pipenv

对于 Python 项目，通过 pipenv 管理包依赖:

pipenv install --python 3.6.5
pipenv install requests

pipenv shell

设置 VSCode 支持 pipenv:

{
    "python.pythonPath": "/Users/fannheyward/.virtualenvs/tools-CDG8SfKX/bin/python"
}

tee

read from stdin, write to stdout AND files.

curl https://github.com/fannheyward.keys | tee -a ~/.ssh/authorized_keys