I finally got the nginx works for WordPress with subdirectory

https://my-website.com/blog

1. Setup a localhost website to wordpress
   […]

2. nginx reverse proxy

   server {
   listen         443 ssl;
   ...
   
   # the trailing '/' is vital
   location ^~ /blog/ {
       proxy_set_header Host $host;
       proxy_set_header X-Forwarded-For $remote_addr;
       proxy_set_header X-Forwarded-Proto https;
       proxy_pass http://localhost:7789/;
   }
   }

3. defines to wp-config.php

   /** set the site URL */
   define('WP_HOME', '/blog');
   define('WP_SITEURL', '/blog');
   $_SERVER['REQUEST_URI'] = str_replace("/wp-admin/", "/blog/wp-admin/",  $_SERVER['REQUEST_URI']);

4. change wp-includes/load.php to support ssl reverse proxy
   […]

5. make sure to disable the chrome disk cache to debug (F12 -> Disable cache)