歪林網誌

本來這個blog是記錄開發輸入法的點滴的,後來越來越雜,現在什麼都記錄了。

2016年1月5日 星期二

Cross compile subversion server (svnserve) from scratch for openwrt:

Unfortunately svnserve is not likely to be availabe in the chaos calmer (15.05) version of of openwrt.
This article describes how to cross compile svnserve (with very basic funtionality) from scratch with the toolchain.
It assumed the "mvebu" toolchain for the router Linksys WRT1200AC / WRT1900AC, but the procedures are similar for other routers.
  1. Download the toolchain for WRT1200AC OpenWrt-Toolchain-mvebu_gcc-5.2.0_musl-1.1.11_eabi.Linux-x86_64.tar.bz2
  2. Download subversion-1.6.23.tar.gz.
  3. Download apr-1.2.12.tar.bz2
  4. Download apr-util-1.2.12.tar.bz2
  5. Download sqlite-amalgamation-3.6.13.tar.gz
  6. Download zlib-1.2.8.tar.gz
  7. Assume working directory is: /home/dev/svn_wrt1200ac.
    Put all source files above in /home/dev/svn_wrt1200ac/archive.
    cd /home/dev/svn_wrt1200ac
  8. Add cross compiler in PATH.
    tar xjvf archive/OpenWrt-Toolchain-mvebu_gcc-5.2.0_musl-1.1.11_eabi.Linux-x86_64.tar.bz2

    export PATH=$PATH:/home/dev/svn_wrt1200ac/OpenWrt-Toolchain-mvebu_gcc-5.2.0_musl-1.1.11_eabi.Linux-x86_64/toolchain-arm_cortex-a9+vfpv3_gcc-5.2.0_musl-1.1.11_eabi/bin/


    Run arm-openwrt-linux-gcc to see if you can run the executable

  9. Decompress the sources files in the right positions:
    tar xzvf archive/subversion-1.6.23.tar.gz
    tar xjvf archive/apr-1.2.12.tar.bz2
    tar xjvf archive/apr-util-1.2.12.tar.bz2
    tar xzvf archive/zlib-1.2.8.tar.gz
    tar xzvf archive/sqlite-amalgamation-3.6.13.tar.gz
    cd subversion-1.6.23
    ln -s ../sqlite-3.6.13/ sqlite-amalgamation
  10. Compile zlib first
    cd ../zlib-1.2.8
    CC=arm-openwrt-linux-gcc ./configure --prefix=/home/dev/svn_wrt1200ac/finalBins/usr
    make install


  11. Compile apr
    cd ../apr-1.2.12/
    ./configure --host=arm-openwrt-linux \
    ac_cv_file__dev_zero="yes" \
    ac_cv_func_setpgrp_void="yes" \
    apr_cv_process_shared_works="yes" \
    apr_cv_mutex_robust_shared="no" \
    apr_cv_tcp_nodelay_with_cork="no" \
    ac_cv_sizeof_struct_iovec="8" \
    apr_cv_mutex_recursive="yes" \
    --prefix=/home/dev/svn_wrt1200ac/finalBins/usr

    patch -p 0 include/apr.h

    Copy and paste the following, and then press Ctrl+D

    @@ -355,10 +355,10 @@
      * to find the logic for this definition search for "ssize_t_fmt" in
      * configure.in.
      */
    -#error Can not determine the proper size for ssize_t
    +#define APR_SSIZE_T_FMT "d"

     /* And APR_SIZE_T_FMT */
    -#error Can not determine the proper size for size_t
    +#define APR_SIZE_T_FMT "d"

     /* And APR_OFF_T_FMT */
     #define APR_OFF_T_FMT APR_INT64_T_FMT

    Start compile.
    make install

  12. Compile apr-util
    cd ../apr-util-1.2.12/
    cd xml/expat
    ./configure --host=arm-openwrt-linux --prefix=/home/dev/svn_wrt1200ac/finalBins/usr
    make install

    cd ../../
    ./configure --host=arm-openwrt-linux --with-apr=/home/dev/svn_wrt1200ac/finalBins/usr \
    --with-expat=/home/dev/svn_wrt1200ac/finalBins/usr \
    --prefix=/home/dev/svn_wrt1200ac/finalBins/usr
    make install

  13. Compile subversion
    cd ../subversion-1.6.23

    ./configure --host=arm-openwrt-linux \
    --with-zlib=/home/dev/svn_wrt1200ac/finalBins/usr \
    --with-apr=/home/dev/svn_wrt1200ac/finalBins/usr \
    --with-apr-util=/home/dev/svn_wrt1200ac/finalBins/usr \
    --prefix=/home/dev/svn_wrt1200ac/finalBins/usr \
    --without-berkeley-db

    make install

  14. Making tarball and transfer to router.
    cd ../finalBins
    tar -czvf svn1.6.23_wrt1200ac_finalBins.tar.gz usr/lib/lib*so* usr/bin

    Transfer the svn1.6.23_wrt1200ac_finalBins.tar.gz to router. Assume /tmp/svn_wrt1200ac_finalBins.tar.gz. e.g.:
    scp svn1.6.23_wrt1200ac_finalBins.tar.gz root@192.168.1.1:/tmp/

    In router, decompress:
    cd /
    tar -xzvf /tmp/svn1.6.23_wrt1200ac_finalBins.tar.gz
To setup svn server:
(Reference: https://forum.openwrt.org/viewtopic.php?id=11693)
mkdir /tmp/svn_repos
svnadmin create --fs-type fsfs /tmp/svn_repos

Added 3 lines to setup svn password:
In /tmp/svn_repos/conf/svnserve.conf, uncomment the folowing 3 lines
anon-access = read
auth-access = write
password-db = passwd

Add in /tmp/svn_repos/conf/passwd, password for "yylam"
To restart svn server is in /etc/rc.local
/usr/bin/svnserve -d -r /tmp/svn_repos

From OpenWrt forum: Looks like the same SVN package from the OpenWRT old_packages git repository. To reactivate the old_packages, add the following line to your feeds.conf file.
src-git old_packages git://git.openwrt.org/packages.git
hen, once you run scripts/feeds update old_packages; scripts/feeds install -a -p old_packages to update your OpenWRT, execute make menuconfig and enable the subversion package under Network --> Version Control Systems and then recompile your OpenWRT. If you don't like to add the old_packages, then either port the subversion package from the old_repository to OpenWRT GitHUB or wait for someone else to do this for you.

2015年1月18日 星期日

How to install cygwin ssh server and ensure auto login (using RSA public key)

How to install cygwin ssh server and ensure auto login (using RSA public key)

(based on http://techtorials.me/cygwin/sshd-configuration/)
  1. In case you want to uninstall a previously installed CYGWIN ssh server:
    (based on http://superuser.com/questions/110726/how-to-uninstall-reinstall-cygwin-to-use-the-sshd)
    In a Cygwin terminal, type the following:  
    # Remove sshd service
    cygrunsrv --stop sshd
    cygrunsrv --remove sshd
    # Delete any sshd or related users (such as cyg_server) from /etc/passwd
    # (use your favorite editor)
    # Delete any sshd or relaged users (such as cyg_server) from the system
    net user sshd /delete
    net user cyg_server /delete
  2. Make sure your windows has a administrator login. Example used is "ylam".
  3. Install CYGWIN or reinstall CYGWIN to have openssh and rsync.
  4. Run Cygwin Terminal
  5. Change the account settings of "ylam" for CYGWIN:
    chmod +r /etc/passwd
    chmod u+w /etc/passwd
    chmod +r /etc/group
    chmod u+w /etc/group
    chmod 755 /var
    touch /var/log/sshd.log
    chmod 664 /var/log/sshd.log
    editrights -l -u ylam
    editrights -a SeAssignPrimaryTokenPrivilege -u ylam
    editrights -a SeCreateTokenPrivilege -u ylam
    editrights -a SeTcbPrivilege -u ylam
    editrights -a SeServiceLogonRight -u ylam
    editrights -l -u ylam

  6. Run ssh-host-config. Type the parts in red below.  
    $ ssh-host-config

    *** Info: Generating missing SSH host keys
    *** Info: Creating default /etc/ssh_config file
    *** Info: Creating default /etc/sshd_config file

    *** Info: StrictModes is set to 'yes' by default.
    *** Info: This is the recommended setting, but it requires that the POSIX
    *** Info: permissions of the user's home directory, the user's .ssh
    *** Info: directory, and the user's ssh key files are tight so that
    *** Info: only the user has write permissions.
    *** Info: On the other hand, StrictModes don't work well with default
    *** Info: Windows permissions of a home directory mounted with the
    *** Info: 'noacl' option, and they don't work at all if the home
    *** Info: directory is on a FAT or FAT32 partition.
    *** Query: Should StrictModes be used? (yes/no)
    yes

    *** Info: Privilege separation is set to 'sandbox' by default since
    *** Info: OpenSSH 6.1. This is unsupported by Cygwin and has to be set
    *** Info: to 'yes' or 'no'.
    *** Info: However, using privilege separation requires a non-privileged account
    *** Info: called 'sshd'.
    *** Info: For more info on privilege separation read /usr/share/doc/openssh/READ
    ME.privsep.
    *** Query: Should privilege separation be used? (yes/no)
    yes
    *** Info: Updating /etc/sshd_config file

    *** Query: Do you want to install sshd as a service?
    *** Query: (Say "no" if it is already installed as a service) (yes/no)
    yes
    *** Query: Enter the value of CYGWIN for the daemon: []
    (Press Enter)
    *** Info: On Windows Server 2003, Windows Vista, and above, the
    *** Info: SYSTEM account cannot setuid to other users -- a capability
    *** Info: sshd requires. You need to have or to create a privileged
    *** Info: account. This script will help you do so.

    *** Info: It's not possible to use the LocalSystem account for services
    *** Info: that can change the user id without an explicit password
    *** Info: (such as passwordless logins [e.g. public key authentication]
    *** Info: via sshd) when having to create the user token from scratch.
    *** Info: For more information on this requirement, see
    *** Info: https://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-nopasswd1

    *** Info: If you want to enable that functionality, it's required to create
    *** Info: a new account with special privileges (unless such an account
    *** Info: already exists). This account is then used to run these special
    *** Info: servers.

    *** Info: Note that creating a new user requires that the current account
    *** Info: have Administrator privileges itself.

    *** Info: No privileged account could be found.

    *** Info: This script plans to use 'cyg_server'.
    *** Info: 'cyg_server' will only be used by registered services.
    *** Query: Do you want to use a different name? (yes/no)
    yes
    *** Query: Enter the new user name:
    ylam
    *** Query: Reenter:
    ylam

    *** Query: Please enter the password for user 'ylam':
    ylam's Window's password
    *** Query: Reenter:
    renter


    *** Info: The sshd service has been installed under the 'sshd'
    *** Info: account. To start the service now, call `net start sshd' or
    *** Info: `cygrunsrv -S sshd'. Otherwise, it will start automatically
    *** Info: after the next reboot.

    *** Info: Host configuration finished. Have fun!

  7. Start the ssh server by:
    net start sshd
     
  8. Modify Windows firewall to allow port 22 traffic.
    (http://diddy.boot-land.net/ssh/files/firewall.htm)

    Control Panel -> Windows Firewall -> Advanced settings -> Inbound Rules (right click) -> New rule
    ...
    Select TCP and enter 22 ....
     
  9. Test the connection:
    ssh -v ylam@localhost
    (Note: "ylam" should be replaced with your login. And a password is prompted)
     
  10. For login without password, at the remote site (For testing, use local site should also be fine), type:
    a) ssh-keygen -t rsa (press enter for everything)
    b) ssh-copy-id -i ~/.ssh/id_rsa.pub abc@localhost
    c) try login again: ssh abc@localhost. You will not need any password.

2014年12月26日 星期五

How to fix the color problem in the idx / sub file?

How to fix the color problem in the idx / sub file?

Sometimes, even the trick of changing the palette:

palette: 000000, 000000, 000000, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, 000000

 in idx file, there are still problem in the subtitle. Using SubResync in VobSub, some of the subtitle, some of subtitle look ugly:




To fix this:
  1. With subresync, save the final.idx / final.sub as DVDMaestro format.

     
  2. You will see final.son, final.spf (palette) and some bitmaps.

     
  3. Open the bitmap file, you will see something like this:


    Open it with image editor (e.g. GIMP)  it is noted it is a bitmap with 16 color palette.
    But only 4 of color palette entries are used.

     

    The 4 colors are: 0x0000ff00 (blue), 0xff000000 (red), 0x00000000 (black) and 0xffffff00 (white)

    We are using this color palette for the subtitle. This color palette must be saved in final.spf.

    With notepad, save the following to a text file final.spf.xxd:

    0000000: 0000 ff00 ff00 0000 0000 0000 ffff ff00 ................
    0000010: ffff ff00 ffff ff00 ffff ff00 ffff ff00 ................
    0000020: ffff ff00 ffff ff00 ffff ff00 ffff ff00 ................
    0000030: ffff ff00 ffff ff00 ffff ff00 ffff ff00 ................

    Using xxd (link) to convert it to final.spf.

    xxd -r < final.spf.xxd > final.spf
     
  4. Fix the final.son  so that the color palette is fixed.
     
    st_format 2
    Display_Start non_forced
    TV_Type PAL
    Tape_Type NON_DROP
    Pixel_Area (0 573)
    Directory .
    Subtitle final
    Display_Area (0 2 719 575)
    Contrast (15 15 15 0)

    SP_NUMBER START END FILE_NAME
    Color (1 3 0 9)
    Contrast (4 15 0 15)

    0001 00:03:02:12 00:03:05:02 final_0001.bmp
    Color (12 15 0 9)
    Contrast (15 15 0 15)

    0002 00:03:05:05 00:03:07:08 final_0002.bmp
    Color (12 2 0 14)
    0003 00:03:07:10 00:03:11:08 final_0003.bmp
    Color (1 3 0 9)
    Contrast (4 15 0 15)
    0004 00:03:11:16 00:03:14:11 final_0004.bmp
    Color (3 2 0 3)
    Contrast (15 15 0 15)
    0005 00:03:15:05 00:03:17:07 final_0005.bmp
    Color (9 2 0 15)
    0006 00:03:17:10 00:03:21:01 final_0006.bmp
    Color (1 3 0 9)
    Contrast (4 15 0 15)
    0007 00:03:21:03 00:03:23:03 final_0007.bmp
     

    The lines in red above caused the wrong subtitle used. So modify it to something like:
     
    st_format 2
    Display_Start non_forced
    TV_Type PAL
    Tape_Type NON_DROP
    Pixel_Area (0 573)
    Directory .
    Subtitle final-2
    Display_Area (0 2 719 575)
    Contrast (15 15 0 15)
    Color (1 2 0 3)

    SP_NUMBER START END FILE_NAME
    0001 00:03:02:12 00:03:05:02 final-2_0001.bmp
    0002 00:03:05:05 00:03:07:08 final-2_0002.bmp
    0003 00:03:07:10 00:03:11:08 final-2_0003.bmp
    0004 00:03:11:16 00:03:14:11 final-2_0004.bmp
    0005 00:03:15:05 00:03:17:07 final-2_0005.bmp
    0006 00:03:17:10 00:03:21:01 final-2_0006.bmp
    0007 00:03:21:03 00:03:23:03 final-2_0007.bmp
    0008 00:03:23:08 00:03:27:00 final-2_0008.bmp
    0009 00:03:27:12 00:03:29:13 final-2_0009.bmp
    0010 00:03:29:15 00:03:30:23 final-2_0010.bmp
    0011 00:03:37:06 00:03:40:14 final-2_0011.bmp
    0012 00:03:40:16 00:03:41:21 final-2_0012.bmp
    0013 00:03:44:09 00:03:47:06 final-2_0013.bmp
    0014 00:03:47:09 00:03:50:01 final-2_0014.bmp
    0015 00:03:50:02 00:03:51:02 final-2_0015.bmp
    0016 00:03:51:04 00:03:53:08 final-2_0016.bmp
    ...

    Note: index 0 is the transparent color.

  5. Download SON2VOBSUB. Open the final.son. Change the height to 576.



    Click Convert.
     
  6. Reopen final.idx with subresync. The subtitle is fixed.

     

 

2014年12月20日 星期六

How to convert TV program to mkv file with subtitle and everything?

This is an example recorded with Magic TV recorder. The movie recorded is "Life of Pi".

  1. Save the recorded program in a USB stick. Copy the *.ts file to d:\temp  and rename it to Pi.ts
     
  2. Use  Handbrake to convert it to a MKV file.

    • De-interlace

       
    • Add all the audio's

       
    • Destination set to "d:\temp\Pi.mkv"
       
    • Start convert.
     
  3. Subtitle handling. Download Project X - DVB demux Tool

    • Drag d:\temp\Pi.ts to it.
       
       
    • Goto PreSettings menu. Choose only to output subtitle:


       
    • Subtitle settings:


       
    • Start convert by clicking "Quick start" and you should get the following files:

      Pi.sup.idx, Pi.sup.sub, Pi-02.sup.idx, Pi-02-sup.sub
       
     
  4. Fix the problem (color and time) of the subtitles. Download VobSub.
     
    • Open SubResync under VobSub. Drag Pi.sup.idx to it. The subtitle color is not correct and the subtitle is not clear.

       
    • Open P1.sup.idx with Notepad. The color index is:
       
      ...
      time offset: 0
      forced subs: OFF
      palette: 000060, 2b2b2b, 2c2c2c, 535353, 353535, 414141, 636363, 7d7d7d, c9c9c9, 8f8f8f, dbdbdb, a9a9a9, ebebeb, 9c9c9c, b8b8b8, 2b2b2b
      custom colors: OFF, tridx: 1000, colors: 600000, 101010, ffffff, a9a9a9
      langidx: 0
      id: --, index: 0

      ....

      By experiment, it is noted that the color marked in red above are the outline of the characters. Suggest to change it to pure black 000000.
      Change the rest to white (ffffffff), i.e.

      palette: 000000, 000000, 000000, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, ffffff, 000000

       
       

    • It is also noted that the timestamp of the subtitle is incorrect.
      Example the first valid subtitle extracted is at 00:09:57.807


      But it actually happens at 00:05:52.500 in the movie (by manually playing the P1.mkv file generated)
      To fix it, double right click the circle of the subtitle item until it becomes a download triangle.
      And type in the correct time.


         
      Save the subtitle as Pi-1-final.idx / sub.

       
    • Do the same to other subtitle Pi-02.sup.idx/sub. Save as Pi-2-final.idx/sub

       
  5. Put the subtitle in the P1mkv file. Download mkvmerge
     
    • Open it. Add Pi-1-final.idx/sub Pi-2-final.idx/sub

       
    • Start muxing as P1-subtitled.mkv
       
    • The final movie has everything:
       

2014年9月21日 星期日

廟街牌匾