In this post, I will record the projects I have done.

2024

Teacher’s Assistant in CS4017 Introduction to Optimization

Keywords

Convex Optimization, CVXPY

Project Description

I was honored to be the TA of Introduction to Optimization. During the semester, I was responsible for holding discussion lectures, office hours, and grading the homeworks and exams. I helped students understand the basic concepts in convex optimization and manipulation of CVXPY.

Project Link

Preliminaries: calculus and linear algebra

Introduction to CVXPY

2023

Communication-Efficient Distributed Optimization with Adaptability to System Heterogeneity

Keywords

Distributed Optimization, ADMM, Heterogeneity, Python

Project Description

We consider the setting of agents cooperatively minimizing the sum of local objectives plus a regularizer on a graph. This paper proposes a primal-dual method in consideration of three distinctive attributes of real-life multi-agent systems, namely: (i) expensive communication, (ii) lack of synchronization, and (iii) system heterogeneity. In specific, we propose a distributed asynchronous algorithm with minimal communication cost, in which users commit variable amounts of local work on their respective subproblems. We illustrate this both theoretically and experimentally in the machine learning setting, where the agents hold private data and use a stochastic Newton method as the local solver. Under standard assumptions on Lipschitz continuous gradients and strong convexity, our analysis establishes linear convergence in expectation and characterizes the dependency of the rate on the number of local iterations. We proceed a step further to propose a simple means for tuning agents’ hyperparameters locally, so as to adjust to heterogeneity and accelerate the overall convergence. Last, we validate our proposed method on a benchmark machine learning dataset to illustrate the merits in terms of computation, communication, and run-time saving as well as adaptability to heterogeneity.

This work was accepted by IEEE Conference on Decision and Control (CDC), 2023.

Project Link

arXiv

IEEE CDC

Project Result

A python (numpy) implementation of linear / logistic regression models, optimized with Gradient Descent / Newton’s method.

Keywords

Linear Regression, Logistic Regression, Gradient Descent, Newton's method, Convex Optimization, Python

Project Description

A python (numpy) implementation of linear / logistic regression models, optimized with Gradient Descent / Newton’s method. An comparison with optimal solution (solved by CVXPY) is also provided.

Project Link

Optimization-Theory

Project Result

2022

Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learning

Keywords

Federated Learning, Zeroth-order Optimization, AirComp, Python

Project Description

Federated learning (FL), as an emerging edge artificial intelligence paradigm, enables many edge devices to collaboratively train a global model without sharing their private data. To enhance the training efficiency of FL, various algorithms have been proposed, ranging from first-order to second-order methods. However, these algorithms cannot be applied in scenarios where the gradient information is not available, e.g., federated black-box attack and federated hyperparameter tuning. To address this issue, in this paper we propose a derivative-free federated zeroth-order optimization (FedZO) algorithm featured by performing multiple local updates based on stochastic gradient estimators in each communication round and enabling partial device participation. Under non-convex settings, we derive the convergence performance of the FedZO algorithm on non-independent and identically distributed data and characterize the impact of the numbers of local iterates and participating edge devices on the convergence. To enable communication-efficient FedZO over wireless networks, we further propose an over-the-air computation (AirComp) assisted FedZO algorithm. With an appropriate transceiver design, we show that the convergence of AirComp-assisted FedZO can still be preserved under certain signal-to-noise ratio conditions. Simulation results demonstrate the effectiveness of the FedZO algorithm and validate the theoretical observations.

This work was accepted by IEEE Transactions on Signal Processing (TSP), 2022.

Project Link

arXiv

IEEE TSP

code

Project Result

2021

Teacher’s Assistant in CS110 Computer Architecture

Keywords

Computer Architecture, RISC-V, C, C++

Project Description

I was honored to be part of the TA group of CS110. During the semester, I was responsible for holding discussion lectures, lab sessions, and office hours. I was also in charge of designing and grading (some of) the homeworks and exams. I helped students understand the basic concepts of computer architecture, such as the RISC-V instruction set, pipelining, and cache. I also helped students with their programming assignments, which were written in C and C++. See the course website for more information: CS110.

Project Link

HW5: CACoin Mining with POSIX Threads: This homework requires students to accelerate the mining process of CACoin by optimizing and parallelizing the code with pthread.

Project 4: Longan Nano Retro Snake Game: This project requires students to implement a retro snake game on the Longan Nano board, using C and RISC-V assembly language. The illustrated game was made by me.

Reproduction of Upper bounds for Model-Free Row-Sparse Principal Component Analysis.

Keywords

Machine Learning, Row-Sparse PCA, Integer Programming, Python

Project Description

This report focus on the reproduction of Upper bounds for Model-Free Row-Sparse Principal Component Analysis.This work proposed a new framework that finds upper (dual) bounds for the sparse PCA within polynomial time via solving a convex integer program (IP). The convex IP is guaranteed to give a dual bound within an affine function of the global optimal value. A monotone local search algorithm is also proposed here to give a good feasible solution for SPCA, which works as the lower (primal) bound of the origin problem. In general, this work presents a complete solution procedure of generating good solutions and proving quality of these solutions. There is currently no comparable theoretical or computational results for solving model-free SPCA comparing to this work.

Project Link

RowSPCA-CS282

Project Result

Implementation of Primal Simplex Method

Keywords

Linear Programming, Classical Simplex, Revised Simplex, Python

Project Description

Simplex method has been a powerful weapon against linear programming problem ever since it was raised. The idea of simplex is to jump from one feasible solution to another, continuously decreasing the objective value until it reaches a global optima, which gives us the optimal feasible solution. Many genres of simplex method have been inspired, including primal classical method, primal revised method and dual simplex method. In this project, I tried to implement primal classical method and primal revised method. Besides, to better start the simplex method, I implemented the two-stage method. A ’Bland’ method is also added here to avoid degenerancy.

Project Link

Primal_Simplex_Solver

Project Result

Pinyin to Hanzi: A Chinese Input Method Based on Statistical Learning

Keywords

NLP, Input Method, Hidden Markov Model, Python

Project Description

Recent years have witnessed sensational progress of Chinese input method driven by some mature techniques, such as dictionary matching, statistical learning and neural networks. The core task of it is translating the raw pinyin series into Chinese characters with valid and reasonable grammar. In this work, we dedicate to develop a demo of Chinese input method by mainly utilizing Hidden Markov Model through three stages: segmentation, tokenization and translation. We verify the empirical performance of our model on test data and explore some insights from the results.

Project Link

PY2HZ

Project Result

2020

Tetrahedralization and Volume Rendering

Keywords

Computer Graphics, Rendering, C++

Project Description

Volume visualization is useful in many areas. We focus on directly rendering unstructured tetrahedral meshed where volume’s interior needs to be visualized. There are many approaches to rendering unstructured grids and accuracy is usually important. Ray-casting based methods, which our work focuses on, are widely accepted. Tetrahedralization means to collect data into multiple tetrahedrons, while some datasets have already implemented this process. Tetrahedralized volume rendering has a better usage of input data and a more efficient calculation especially in the transparent area. Whereas its intuition and procedure are very much familiar to the former one’s.

Project Link

Tetrahedron-Volume-Rendering

Project Result

本文记录了我在学习机器学习、优化、计算机算法等方面看到的好的文章或博文，实时更新。一般直接附上原文链接。

机器学习

教程

• 图解机器学习

决策树

• 决策树（上）——ID3、C4.5、CART

• 决策树（中）——Random Forest、Adaboost、GBDT

• 决策树（下）——XGBoost、LightGBM

深度学习

Neural Tangent Kernal

• Understanding the Neural Tangent Kernel

• Some Math behind Neural Tangent Kernel

优化

凸优化

• Strong convexity
• Lipschitz continuous gradient

计算机算法

算法讲解网站

• OI Wiki: 编程竞赛知识整合，比较详细，适合耐心看。

• Hello算法: 常用算法图解，适合快速回顾。

• 算法通关手册: Python实现，适合刷题背板子。

• 代码随想录: 与算法通关手册互为补充，含多语言版本。

刷题网站

• 牛客网: 需要开会员，大多数在线机考均在牛客上。

• LeetCode Hot 100: 面试前必刷。

写在前面

这应该算是我第一篇正式blog，心情有点小激动。

确定了研究生去向以后心情比较轻松，最近实验室压力也不大（我比较摸鱼），所以想捣鼓这些小东西。
之前花了几天时间建站，过两天应该会再更一个blog复盘建站的过程。

这篇blog是随搭随更新的。网上的资料并不完全一致。至于能不能搭成功，我自己心里也没底。

搭建经过

硬件准备

• Raspberry Pi 4B, 已安装 Raspbian GNU/Linux 10 (buster) armv7l 系统（及其电源线）
• TP-Link AC1200 路由器，型号 TL-WDR5610（及其电源线）
• Seagate SRD0NF1 1T 移动硬盘 （及其数据线）
• 两根网线
• 一台电脑（Win10）

路由器安装

由于我自购了一个路由器（人傻钱多），第一步当然是安装路由器了。接下来步骤和说明书上一致，不想看可以跳过。

连接路由器

• 一根网线连接WAN口和网线接口
• 连接路由器电源
• 电脑通过无线连接来连接路由器（默认名称一般在路由器底部或背后）
• 检查指示灯

设置路由器

注意要先关闭代理

• 打开设置网址，TP-Link是tplogin.cn
• 创建管理员密码
• 上网设置，给我检测的是自动获得IP地址，注意是WAN口IP地址
• 无线设置，设置2.4/5G频段，设置WIFI名称和密码

以后管理连接该路由器的设备就可以在设置网址中进行了。

树莓派远程连接

我们需要在电脑上远程连接树莓派，这里我选择的是MobaXterm。官网链接

树莓派连接路由器

• 树莓派连接电源线
• 树莓派连接网线，网线另一端连接路由器的一个LAN口上

电脑SSH连接树莓派

• 在MobaXterm中通过SSH连接树莓派，树莓派IP地址在 tplogin.cn 可以看到
• 当然我这边采用了RDP协议，因为我本身在树莓派中安装的系统是带图形界面的

树莓派设置静态IP

这一部分是抄 https://zhuanlan.zhihu.com/p/57202972 的。我之前也看到过其他设置静态IP的方法，暂不清楚这个方法行不行得通。

• 打开DHCP配置文件

  $ sudo nano /etc/dhcpcd.conf

• 在该文件中添加/修改以下几项

  #指定接口eth0 interface eth0 # 指定静态IP， /24表示子网掩码为 255.255.255.0 static ip_address=192.168.1.101/24 # 路由器/网关IP地址 static routers=192.168.1.1

• 保存并关闭该文件，随后重启树莓派

  $ sudo reboot

挂载移动硬盘

• 连接移动硬盘至树莓派
• 查看是否连接成功

  $ df -hT

可以看到列表中出现了一行

Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/sda1      fuseblk   932G  148M  932G   1% /media/root/Seagate Portable Drive

我们看到这边硬盘格式是fuseblk，据某论坛, 我们暂且认为
这边fuseblk == NTFS。

挂载硬盘

• 挂载硬盘

  $ sudo mkdir /home/pi/seagate $ sudo mount /dev/sda1 /home/pi/seagate

  如果在挂载时出现如下错误

  Mount is denied because the NTFS volume is already exclusively opened. The volume may be already mounted, or another software may use it which could be identified for example by the help of the 'fuser' command.

  $ fuser -m /dev/sda1

  发现

  /dev/sda1: 1425

  那就杀死TA！

  $ kill 1425

  再次进行mount操作

开机自动挂载

大佬说 NTFS 格式 Linux 不能直接读取（没有求证），下面解决这一问题。参考网址

• 安装NTFS格式可读写软件

  $ sudo apt-get install ntfs-3g $ modprobe fuse

• 硬盘开机自动挂载

  $ sudo nano /etc/fstab

  并在最后一行添加

  /dev/sda1 /home/pi/seagate fuseblk defaults,noexec,umask=0000 0 0

Update!!: 昨晚临走的时候直接拔了树莓派电源。今天重新打开的时候，发现它进入了emergency mode。经过了一波搜索，我发现了链接问题比较相似。于是就把昨天添加的一行修改为

/dev/sda1 /home/pi/seagate ntfs defaults,noexec,_netdev,umask=0000 0 0

reboot后开机成功

部署Seafile

安装seafile

• 更新源、安装python环境

  $ sudo apt-get update $ sudo apt-get upgrade $ sudo apt-get install python2.7 libpython2.7 python-setuptools python-pil python-ldap python-urllib3 sqlite3 python-requests

• 下载安装包，可以在树莓派通过wget下载，也可以电脑下载后传到树莓派里。这边我选择后者

• 在github中下载了seafile-server_7.0.5_stable_pi.tar.gz。链接

• 树莓派中在 /home/pi/seafile目录下解压

  $ tar -xzf seafile-server_7.0.5_stable_pi.tar.gz

• 进行安装

  $ bash setup-seafile.sh

• 设置server信息

# 服务器名称
[server name]: noobBran
# 局域网内树莓派IP
[This server's ip or domain]: 192.168.1.101
# 指定seafile数据存放位置，我们将其改到硬盘目录下
[default: /home/pi/seafile/seafile-data ] /home/pi/seagate/seafile-data
# seafile服务器端口号，使用默认8082
[default: 8082 ] 

注意!!: 如果出现如下报错

Failed to sync seafile database

在setup-seafile.sh中进行修改

- seafile_db=${TOPDIR}/seafile-data/seafile.db #删除这一行
+ seafile_db=${seafile_data_dir}/seafile.db #添加这一行（没有前面 + 号）

随后重新安装即可。出现以下字样即安装成功

-----------------------------------------------------------------
Your seafile server configuration has been completed successfully.
-----------------------------------------------------------------

run seafile server:     ./seafile.sh { start | stop | restart }
run seahub  server:     ./seahub.sh  { start <port> | stop | restart <port> }

-----------------------------------------------------------------
If the server is behind a firewall, remember to open these tcp ports:
-----------------------------------------------------------------

port of seafile fileserver:   8082
port of seahub:               8000

开启seafile

• 开启服务

  bash seafile.sh start && bash seahub.sh start

  并配置管理员邮箱和密码

打开浏览器并访问http://localhost:8000

• 设置开机自启动。在seafile-server_7.0.5目录下建一个seafile_start.sh。并填写

  sudo bash seafile.sh start sudo bash seahub.sh start

• 打开/etc/rc.local，在exit 0前添加

  sudo bash /home/pi/seafile/seafile-server_7.0.5/seafile_start.sh

seafile局域网访问失败

• 目前其他机器通过局域网访问失败。下面解决这一问题
• 打开seafile/conf/gunicorn.conf

  - bind = "127.0.0.1:8000" # 删除这一行 + bind = "192.168.1.101:8000" # 添加这一行

• 重启seafile

  sudo bash seafile.sh restart sudo bash seahub.sh restart

其他设备连接私有云

前提条件：同一局域网下

1. 浏览器输入http://192.168.1.101:8000
2. 下载客户端。下载链接: https://www.seafile.com/download/

成果展示

Reference

https://blog.csdn.net/windsnow1/article/details/96281071/

https://cloud.tencent.com/developer/article/1681154

https://zhuanlan.zhihu.com/p/57202972

https://blog.csdn.net/qq_34178764/article/details/104404738?utm_medium=distribute.pc_relevant.none-task-blog-2defaultbaidujs_baidulandingword~default-0.control&spm=1001.2101.3001.4242

https://blog.csdn.net/windsnow1/article/details/96281071/

https://www.kancloud.cn/kancloud/seafile-manual/51506