Subspace Institute
Background Decoration

Eye of Providence

プロビデンスの目

Intro

Full-featured social media monitor that extracts data from a variety of services and pushes updates to Telegram or other platforms

Features

  • Monitor several services at the same time
  • Support retry on failed connections
  • Proxy support to avoid API rate limit
  • Low memory footprint (About 50 MB for a single account, 120 MB for 20 accounts with multiple services)
  • ESM by default with minimal dependencies

Supported Services (and plans)

  • bilibili
  • bilibili-live
  • douyin (cookies required)
  • douyin-live (cookies required)
  • twitch (API required)
  • instagram
  • tiktok (cookies required, unstable)
  • tiktok-live
  • twitter (via rsshub-json)
  • weibo (cookies required for follower-only activities)
  • xiaohongshu (cookies required for per-note links + publish timestamps)
  • youtube (via rsshub-json)
  • youtube-live
  • rss (supports RSSHub and rsshub-json
  • github (via rsshub-json)
  • tapechat
  • afdian
  • netease-music (via NeteaseCloudMusicApi)

Supported Senders

  • telegram

System Requirements

  • Node.js >= 18

Usage

docker run --init \
  -v $(pwd)/config.js:/app/config.js:ro \
  -v $(pwd)/db:/app/db \
  ghcr.io/laplace-live/eop -c config.js --color

Configurations

Minimal config.js:

config.js
export default {
  accounts: [
    {
      enabled: true,
      slug: "嘉然",
      biliId: "672328094",
    },
  ],
};

Your full config.js configuration may look like:

config.js
export default {
  // Loop interval in milliseconds
  loopInterval: 60 * 1000,

  // A small amount of time to wait inserted before each account
  loopPauseTimeBase: 1000,

  // Math.random() time factor for `loopPauseTimeBase`
  loopPauseTimeRandomFactor: 2000,

  // 24 hours, if latest post older than this value, do not send notifications
  douyinBotThrottle: 36 * 3600 * 1000,
  douyinLiveBotThrottle: 1200 * 1000, // 20 mins

  // 65 mins, bilibili sometimes got limit rate for 60 mins.
  bilibiliBotThrottle: 65 * 60 * 1000,
  bilibiliLiveBotThrottle: 65 * 60 * 1000,
  bilibiliFollowingBotThrottle: 3600 * 1000,

  rssBotThrottle: 12 * 3600 * 1000,
  weiboBotThrottle: 3600 * 1000,
  tapechatBotThrottle: 3600 * 1000,
  afdianBotThrottle: 3600 * 1000,
  twitchBotThrottle: 65 * 60 * 1000,

  // Custom proxy to bypass bilibili API rate limit
  // Default: ''
  rateLimitProxy: "http://10.2.1.2:7890",

  // Per-request timeout in milliseconds (applied via `AbortSignal.timeout()`)
  // Default: 4000
  pluginOptions: {
    requestTimeout: 3000,
    // Cookies are no longer configured here — supply them via CookieCloud or
    // `COOKIES_*` environment variables (see "Environment Variables" below).
  },

  // Twitch API credentials
  // Create your twitch app: https://dev.twitch.tv/console
  twitch: {
    clientId: `xxx`,
    clientSecret: `yyy`,
  },

  // Telegram global configs
  telegram: {
    enabled: true,
    apiBase: "https://api.telegram.org/bot",
    token: "",

    // If bot belongs to a premium account. Accounts with premium perks has
    // 2048 characters limit (1024 for free accounts) for photo and video
    // captions
    // Defautl: false
    premium: true,

    // Define a special channel / chat to send debugging info. If this is not
    // defined. No debugging info will be sent.
    // Default: undefined
    debuggingChannelId: -10012345,
  },

  // Proxy to fetch images
  // For more info: https://github.com/willnorris/imageproxy
  // Default: ''
  imageProxy: `https://myimageproxy.tld/imageproxy/`,

  accounts: [
    {
      // Use `false` to disable checking this profile
      // Default: true
      enabled: true,

      // Slug is used to identify accounts in logs
      slug: "嘉然",

      // Set to `true` to add `slug` at the beginning of the notification.
      // ie: #嘉然. Useful for pushing notifications with multiple accounts in
      // one channel
      // Default: false
      showSlug: true,

      // When `showSlug` is `true` and this option is defined. `slugNotification`
      // will be used instead of `slug`. This can be helpful when you have
      // multiple accounts but still want to show the same notification slug.
      // For example account A and B must be different slugs but when you set
      // `slugNotification`. They can be the same in the notifications.
      slugNotification: "嘉然",

      // bilibili account UID
      biliId: "672328094",

      // Use real `.bilibili.com` cookies (from CookieCloud / `COOKIES_BILIBILI`)
      // for all bilibili read paths (dynamics, following, live). When off,
      // requests use an anonymous random `buvid3` (no login). Enable if you need
      // logged-in-only data or keep hitting risk control (风控) while anonymous.
      // Default: false
      bilibiliEnableCookieCloud: true,

      // Check bilibili activity comments. Disabled by default
      // This fires another API to monitor comments and replies. It's not
      // recommended to enable this feature if you have a lot of accounts to
      // monitor or you will hit API rate limit soon.
      // Default: false
      bilibiliFetchComments: true,

      // In addition to `bilibiliFetchComments`, this also fetches the comments
      // and replies in the sticky dynamic
      // Default: false
      bilibiliFetchCommentsSticky: true,

      // How many page to fetch for comments. Should be >= 0.
      // 0 means fetch only 1 (index 0) page
      // Default: 5
      bilibiliFetchCommentsLimit: 5,

      // In addition to `bilibiliFetchComments`, this disables fetching replies
      // in each comments.
      // Default: false
      bilibiliFetchCommentsDisableReplies: true,

      // Fetch specific users from comments and replies
      // Default: undefined
      bilibiliFetchCommentsWatchUsers: ["2132180406"],

      // Detect comments and replies from the following list. It only works when
      // the user has public following list.
      // Default: false
      bilibiliFetchCommentsFromFollowing: true,

      // Detect comments and replies from public VUP database
      bilibiliFetchCommentsFromVupList: true,

      // Check bilibili account following. Disabled by default
      // This fires another API to monitor account following. It's not
      // recommended to enable this feature if you have a lot of accounts to
      // monitor or you will hit API rate limit soon.
      // Default: false
      bilibiliFetchFollowing: true,

      // Enable this to fetch users with private following enabled
      // Default: false
      bilibiliFetchFollowingDeprecatedApi: true,

      // Whether to check fans medal wear on/off states. This is disabled by
      // default because of the API is unstable and you may got many false
      // positive results.
      // Default: false
      bilibiliFansMedalOnOff: true,

      // Douyin account ID
      douyinId: "MS4wLjABAAAA5ZrIrbgva_HMeHuNn64goOD2XYnk4ItSypgRHlbSh1c",

      // Douyin live ID is separated and need to be calculated from `douyinId`
      douyinLiveId: "",

      // Weibo account ID
      weiboId: "7595006312",

      // Check Weibo activity comments. Disabled by default
      // This fires another API to monitor comments and replies. It's not
      // recommended to enable this feature if you have a lot of accounts to
      // monitor or you will hit API rate limit soon.
      // Default: false
      weiboFetchComments: true,

      // In addition to `weiboFetchComments`, this also fetches the comments
      // and replies in the sticky statuses
      // Default: false
      weiboFetchCommentsSticky: true,

      // Fetch specific users from comments and replies
      // Default: undefined
      weiboFetchCommentsWatchUsers: ["12345"],

      // RSS services. One account can have more than one RSS service. They will
      // be executed one by one in the loop.
      rss: [
        {
          // Name will be used as tag in notification output
          name: "Twitter",

          // Slug stored in database. Must be unique for current account (Can
          // not be `bilibili`, `weibo`, or other predefined services)
          slug: "rss_twitter",

          // Can be 'twitter', 'youtube', or 'rss' for general RSS feeds
          type: "twitter",

          // Language tag used by timestamp
          lang: "ja",

          // Can be `rsshub-json` or `rsshub`
          provider: "rsshub-json",

          // Use `imageProxy` to fetch images
          useImageProxy: true,

          // Regex to match for title. Only items match the regex will be sent
          titleRegex: /posted a comment/,

          // Same as `titleRegex` but for content. Please note that when both
          // options are defined, notification will only be sent when both of
          // them match.
          contentRegex: /^7-Zip ([v0-9.]*) was released/,

          url: "http://rsshub-json-instance/twitter/user/minatoaqua/showAuthorInDesc=1&showEmojiForRetweetAndReply=1&showRetweetTextInTitle=0&showQuotedInTitle=1&heightOfPics=150",

          // Optional extra request headers sent with this feed's fetch. Use to
          // clear an origin's Cloudflare WAF / bot protection via a custom-rule
          // header (configure a matching "skip"/allow rule on your edge). The
          // value can come from an env var so the secret stays out of config.
          headers: { "x-service-waf": process.env.RSS_TWITTER_BYPASS },
        },
        {
          name: "YouTube動画",
          slug: "rss_youtube",
          type: "youtube",
          lang: "ja",
          provider: "rsshub",
          url: "https://rsshub.app/youtube/channel/UC1opHUrw8rvnsadT-iGp7Cg",
        },
      ],

      // Tape message box account ID. Usually the last part of your message
      // box's URL. ie. https://www.tapechat.net/uu/TDL6BG/EVWKIS0F the
      // `tapechatId` should be `EVWKIS0F`
      tapechatId: "RQOPYMJQ",

      // Aifadian (afdian) user ID, can be acquired from any API requests like:
      // https://afdian.com/api/badge/first_show?creator_id=xxx
      afdianId: "beaf1482bc2511ea896452540025c377",

      // Telegram chat/channel ID to receive notifications
      tgChannelId: 41205411,

      // Update Telegram chat/channel photo/avatar when user avatar updates in
      // included sources.
      // Default: []
      tgChannelAvatarSource: ["weibo", "bilibili"],

      // Show custom color output in console. Nothing useful
      // Default: '#fff'
      color: "#e799b0",

      // Avoid chekcing bilibili live stream. Some accounts may not have live
      // stream ability
      // Default: false
      disableBilibiliLive: false,

      // Avoid checking douyin live stream
      // Default: false
      disableDouyinLive: false,
    },
    {
      enabled: true,
      slug: "贝拉",
      showSlug: true,
      biliId: "672353429",
      douyinId:
        "MS4wLjABAAAAlpnJ0bXVDV6BNgbHUYVWnnIagRqeeZyNyXB84JXTqAS5tgGjAtw0ZZkv0KSHYyhP",
      douyinLiveId: "820648166099",
      weiboId: "7594710405",
      tgChannelId: "41205411",
      color: "#bd7d74",
    },
  ],
};

Environment Variables

Secrets and host-level settings are read from the process environment (e.g. a .env file, or your container's env) rather than from config.js, which holds only non-secret, per-account options.

Cookies & authentication

The preferred way to supply cookies is CookieCloud, which syncs cookies (including HttpOnly ones) from your browser to a self-hosted server. eop fetches and decrypts them once per loop and matches them by domain.

VariableDescription
COOKIE_CLOUD_HOSTCookieCloud server URL, e.g. https://cookies.example.com. All three COOKIE_CLOUD_* vars must be set to enable syncing.
COOKIE_CLOUD_UUIDCookieCloud user key / UUID.
COOKIE_CLOUD_PASSWORDCookieCloud end-to-end password.
COOKIES_DOUYINDouyin cookie string (needs __ac_nonce + __ac_signature).
COOKIES_WEIBO_WEBWeibo .weibo.com SUB/SUBP cookie. Used by both Weibo tiers (raw fetch + CloakBrowser fallback).
COOKIES_BILIBILIbilibili cookie string (e.g. SESSDATA=...).
COOKIES_XIAOHONGSHUxiaohongshu logged-in cookie (web_session + a1) — enables per-note links, publish timestamps, and avoids 风控.
COOKIES_<SLUG>Per-RSS-feed cookie, where <SLUG> is the feed's slug upper-cased (e.g. COOKIES_RSS_XIAOHONGSHU).

For each service, cookies resolve in order — first non-empty wins:

  1. CookieCloud (matched by domain)
  2. the matching COOKIES_* variable above

bilibili is opt-in per account. It uses the resolved cookie only when the account sets bilibiliEnableCookieCloud (default off); while off it uses an anonymous randomly generated cookie, so COOKIE_CLOUD_* and COOKIES_BILIBILI are ignored for that account until you enable the toggle. Douyin, Weibo, and xiaohongshu always use the resolved cookie.

Senders

VariableDescription
TELEGRAM_TOKENTelegram bot token. Used as the fallback when telegram.token is not set in config.js.

Weibo CloakBrowser fallback

When the raw Weibo fetch is blocked, eop retries the same endpoints inside a remote CloakBrowser (cloakserve) session. The fallback is only active when CLOAKBROWSER_HOST is set.

VariableDescription
CLOAKBROWSER_HOSTcloakserve endpoint, e.g. http://127.0.0.1:3000. Unset = browser fallback disabled (raw fetch only).
CLOAKBROWSER_FINGERPRINT_POOL_SIZENumber of reusable browser fingerprint seeds. Default: 6.

Networking & misc

VariableDescription
HTTP_PROXYProxy used when fetching/processing remote media.
EOP_DEFAULT_USER_AGENTOverrides the default desktop User-Agent used for requests.

Testing

The test suite gates live/integration tests behind extra variables (unset = those tests are skipped):

VariableDescription
EOP_INTEGRATION_TESTSEnables bilibili integration tests.
COOKIE_CLOUD_HOSTAlso gates live Weibo tests (seeds a real .weibo.com SUB from CookieCloud).
CLOAKBROWSER_HOSTGates the live CloakBrowser e2e test (in addition to COOKIE_CLOUD_HOST).
DOUYIN_COOKIES / TIKTOK_COOKIESCookies for the Douyin / TikTok extractor tests.
TWITCH_CLIENT_ID / TWITCH_CLIENT_SECRETCredentials for the Twitch extractor test.

Development

You need to have Bun installed first:

# Install dependencies
bun install

# Create config file
vi config.js

# Execute locally
bun run src/index.js run -c config.js --verbose --once

# ...or via the start script
bun start -c config.js --verbose --once

# Run tests
bun test

Build and run the local Docker image:

# Build the `laplace-live/eop:local` image (see docker-bake.hcl)
docker buildx bake

docker run --init -it \
  -v (pwd)/config.js:/app/config.js:ro \
  -v (pwd)/db:/app/db laplace-live/eop:local \
  -c config.js --color --verbose

FAQ

Why this name?

The original intention of this project was to monitor updates of a Chinese VTuber group A-SOUL. Now this project has been renamed to eop for more general use.

Why not executing checks in parallel?

Most services have API limits or rate limits. Executing checks in parallel only make sense with small amount of accounts.

License

AGPL-3.0

On this page