Subspace Institute

Eye of Providence

プロビデンスの目

Intro

Full-featured social media monitor that extracts data from a variety of services and pushes updates to Telegram or other platforms.

Features

  • Monitor several services at the same time
  • Support retry on failed connections
  • Proxy support to avoid API rate limit
  • Low memory footprint (About 50 MB for a single account, 120 MB for 20 accounts with multiple services)
  • ESM by default with minimal dependencies

Supported Services (and plans)

  • bilibili
  • bilibili-live
  • douyin (cookies required)
  • douyin-live (cookies required)
  • twitch (API required)
  • instagram
  • tiktok (cookies required, unstable)
  • tiktok-live
  • twitter (via rsshub-json)
  • weibo (cookies required for follower-only activities)
  • youtube (via rsshub-json)
  • youtube-live
  • rss (supports RSSHub and rsshub-json)
  • github (via rsshub-json)
  • ddstats
  • tapechat
  • afdian
  • qq-music (cookies required, need rotate manually)
  • netease-music (via NeteaseCloudMusicApi)

Supported Senders

  • telegram
  • go-cqhttp (QQ Guilds/Groups) - Not actively maintained

System Requirements

  • Node.js >= 18

Configurations

Minimal config.js:

config.js
export default {
  accounts: [
    {
      enabled: true,
      slug: '嘉然',
      biliId: '672328094',
    },
  ],
}

Your full config.js configuration may look like:

config.js
export default {
  // Loop interval in milliseconds
  loopInterval: 60 * 1000,
 
  // A small amount of time to wait inserted before each account
  loopPauseTimeBase: 1000,
 
  // Math.random() time factor for `loopPauseTimeBase`
  loopPauseTimeRandomFactor: 2000,
 
  // 24 hours, if latest post older than this value, do not send notifications
  douyinBotThrottle: 36 * 3600 * 1000,
  douyinLiveBotThrottle: 1200 * 1000, // 20 mins
 
  // 65 mins, bilibili sometimes got limit rate for 60 mins.
  bilibiliBotThrottle: 65 * 60 * 1000,
  bilibiliLiveBotThrottle: 65 * 60 * 1000,
  bilibiliFollowingBotThrottle: 3600 * 1000,
 
  rssBotThrottle: 12 * 3600 * 1000,
  weiboBotThrottle: 3600 * 1000,
  ddstatsBotThrottle: 3600 * 1000,
  tapechatBotThrottle: 3600 * 1000,
  afdianBotThrottle: 3600 * 1000,
  qqMusicBotThrottle: 3600 * 1000,
  twitchBotThrottle: 65 * 60 * 1000,
 
  // Custom proxy to bypass bilibili API rate limit
  // Default: ''
  rateLimitProxy: 'http://10.2.1.2:7890',
 
  // Options for got
  // Default:
  // {
  //   requestOptions: {
  //     timeout: {
  //       request: 4000
  //     }
  //   }
  // }
  pluginOptions: {
    requestOptions: {
      timeout: {
        request: 3000,
      },
    },
    customCookies: {
      // Nov 11, 2021
      // Douyin main site now requires `__ac_nonce` and `__ac_signature` to work
      douyin: `__ac_nonce=XXX; __ac_signature=XXX;`,
 
      // get `SESSDATA` cookie from https://www.bilibili.com/
      bilibili: `SESSDATA=XXX`,
 
      // get `SUB` cookie from https://m.weibo.cn/
      weibo: `SUB=XXX`,
    },
  },
 
  // Twitch API credentials
  // Create your twitch app: https://dev.twitch.tv/console
  twitch: {
    clientId: `xxx`,
    clientSecret: `yyy`,
  },
 
  // Telegram global configs
  telegram: {
    enabled: true,
    apiBase: 'https://api.telegram.org/bot',
    token: '',
 
    // If bot belongs to a premium account. Accounts with premium perks has
    // 2048 characters limit (1024 for free accounts) for photo and video
    // captions
    // Defautl: false
    premium: true,
 
    // Define a special channel / chat to send debugging info. If this is not
    // defined. No debugging info will be sent.
    // Default: undefined
    debuggingChannelId: -10012345,
  },
 
  // QQ Guild global configs
  qGuild: {
    enabled: true,
    // go-cqhttp endpoint
    // See https://github.com/Mrs4s/go-cqhttp to learn how to deploy qo-cqhttp
    // and send updates to QQ Guild
    apiBase: 'http://10.2.1.2:5700',
  },
 
  // Sentry global configs
  sentry: {
    enabled: true,
    dsn: `https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx@ingest.sentry.io/9`,
    environment: `production`,
    tracesSampleRate: 1.0,
  },
 
  // Proxy to fetch images
  // For more info: https://github.com/willnorris/imageproxy
  // Default: ''
  imageProxy: `https://myimageproxy.tld/imageproxy/`,
 
  accounts: [
    {
      // Use `false` to disable checking this profile
      // Default: true
      enabled: true,
 
      // Slug is used to identify accounts in logs
      slug: '嘉然',
 
      // Set to `true` to add `slug` at the beginning of the notification.
      // ie: #嘉然. Useful for pushing notifications with multiple accounts in
      // one channel
      // Default: false
      showSlug: true,
 
      // When `showSlug` is `true` and this option is defined. `slugNotification`
      // will be used instead of `slug`. This can be helpful when you have
      // multiple accounts but still want to show the same notification slug.
      // For example account A and B must be different slugs but when you set
      // `slugNotification`. They can be the same in the notifications.
      slugNotification: '嘉然',
 
      // bilibili account UID
      biliId: '672328094',
 
      // Check bilibili activity comments. Disabled by default
      // This fires another API to monitor comments and replies. It's not
      // recommended to enable this feature if you have a lot of accounts to
      // monitor or you will hit API rate limit soon.
      // Default: false
      bilibiliFetchComments: true,
 
      // In addition to `bilibiliFetchComments`, this also fetches the comments
      // and replies in the sticky dynamic
      // Default: false
      bilibiliFetchCommentsSticky: true,
 
      // How many page to fetch for comments. Should be >= 0.
      // 0 means fetch only 1 (index 0) page
      // Default: 5
      bilibiliFetchCommentsLimit: 5,
 
      // In addition to `bilibiliFetchComments`, this disables fetching replies
      // in each comments.
      // Default: false
      bilibiliFetchCommentsDisableReplies: true,
 
      // Fetch specific users from comments and replies
      // Default: undefined
      bilibiliFetchCommentsWatchUsers: ['2132180406'],
 
      // Detect comments and replies from the following list. It only works when
      // the user has public following list.
      // Default: false
      bilibiliFetchCommentsFromFollowing: true,
 
      // Detect comments and replies from public VUP database
      bilibiliFetchCommentsFromVupList: true,
 
      // Check bilibili account following. Disabled by default
      // This fires another API to monitor account following. It's not
      // recommended to enable this feature if you have a lot of accounts to
      // monitor or you will hit API rate limit soon.
      // Default: false
      bilibiliFetchFollowing: true,
 
      // Enable this to fetch users with private following enabled
      // Default: false
      bilibiliFetchFollowingDeprecatedApi: true,
 
      // Whether to check fans medal wear on/off states. This is disabled by
      // default because of the API is unstable and you may got many false
      // positive results.
      // Default: false
      bilibiliFansMedalOnOff: true,
 
      // Douyin account ID
      douyinId: 'MS4wLjABAAAA5ZrIrbgva_HMeHuNn64goOD2XYnk4ItSypgRHlbSh1c',
 
      // Douyin live ID is separated and need to be calculated from `douyinId`
      douyinLiveId: '',
 
      // Weibo account ID
      weiboId: '7595006312',
 
      // Check Weibo activity comments. Disabled by default
      // This fires another API to monitor comments and replies. It's not
      // recommended to enable this feature if you have a lot of accounts to
      // monitor or you will hit API rate limit soon.
      // Default: false
      weiboFetchComments: true,
 
      // In addition to `weiboFetchComments`, this also fetches the comments
      // and replies in the sticky statuses
      // Default: false
      weiboFetchCommentsSticky: true,
 
      // Fetch specific users from comments and replies
      // Default: undefined
      weiboFetchCommentsWatchUsers: ['12345'],
 
      // QQ Music ID
      qqMusicId: 'oKCPNKnPoinAoz**',
 
      // QQ Music channel ID. Can only be aquired from single post URL.
      // Share the single post from the app to get it.
      qqMusicChannelId: '10036163',
 
      // RSS services. One account can have more than one RSS service. They will
      // be executed one by one in the loop.
      rss: [
        {
          // Name will be used as tag in notification output
          name: 'Twitter',
 
          // Slug stored in database. Must be unique for current account (Can
          // not be `bilibili`, `weibo`, or other predefined services)
          slug: 'rss_twitter',
 
          // Can be 'twitter', 'youtube', or 'rss' for general RSS feeds
          type: 'twitter',
 
          // Language tag used by timestamp
          lang: 'ja',
 
          // Can be `rsshub-json` or `rsshub`
          // See https://github.com/sparanoid/rsshub-json for more info
          provider: 'rsshub-json',
 
          // Use `imageProxy` to fetch images
          useImageProxy: true,
 
          // Regex to match for title. Only items match the regex will be sent
          titleRegex: /posted a comment/,
 
          // Same as `titleRegex` but for content. Please note that when both
          // options are defined, notification will only be sent when both of
          // them match.
          contentRegex: /^7-Zip ([v0-9.]*) was released/,
 
          url: 'http://rsshub-json-instance/twitter/user/minatoaqua/showAuthorInDesc=1&showEmojiForRetweetAndReply=1&showRetweetTextInTitle=0&showQuotedInTitle=1&heightOfPics=150',
        },
        {
          name: 'YouTube動画',
          slug: 'rss_youtube',
          type: 'youtube',
          lang: 'ja',
          provider: 'rsshub',
          url: 'https://rsshub.app/youtube/channel/UC1opHUrw8rvnsadT-iGp7Cg',
        },
      ],
 
      // Tape message box account ID. Usually the last part of your message
      // box's URL. ie. https://www.tapechat.net/uu/TDL6BG/EVWKIS0F the
      // `tapechatId` should be `EVWKIS0F`
      tapechatId: 'RQOPYMJQ',
 
      // Aifadian (afdian) user ID, can be acquired from any API requests like:
      // https://afdian.net/api/badge/first_show?creator_id=xxx
      afdianId: 'beaf1482bc2511ea896452540025c377',
 
      // Telegram chat/channel ID to receive notifications
      tgChannelId: 41205411,
 
      // QQ guild ID to receive notifications
      qGuildId: '12345678901234567',
 
      // QQ guild channel ID to receive notifications, `qGuildId` is also
      // required to identify which channel to be sent
      qGuildChannelId: 1234567,
 
      // Update Telegram chat/channel photo/avatar when user avatar updates in
      // included sources.
      // Default: []
      tgChannelAvatarSource: ['weibo', 'bilibili'],
 
      // Show custom color output in console. Nothing useful
      // Default: '#fff'
      color: '#e799b0',
 
      // Avoid chekcing bilibili live stream. Some accounts may not have live
      // stream ability
      // Default: false
      disableBilibiliLive: false,
 
      // Avoid checking douyin live stream
      // Default: false
      disableDouyinLive: false,
 
      // Disable checking DDStats. Some bilibili accounts may not have DDStats
      // feature enabled
      // Default: false
      disableDdstats: false,
    },
    {
      enabled: true,
      slug: '贝拉',
      showSlug: true,
      biliId: '672353429',
      douyinId: 'MS4wLjABAAAAlpnJ0bXVDV6BNgbHUYVWnnIagRqeeZyNyXB84JXTqAS5tgGjAtw0ZZkv0KSHYyhP',
      douyinLiveId: '820648166099',
      weiboId: '7594710405',
      tgChannelId: '41205411',
      color: '#bd7d74',
    },
  ],
}

CookieCloud Support

This project supports CookieCloud to sync cookies from your local instance.

Create the following environment variables to enable CookieCloud:

.env
COOKIE_CLOUD_HOST=https://url
COOKIE_CLOUD_UUID=<uuid>
COOKIE_CLOUD_PASSWORD=<password>

Development

You need to have PNPM installed first:

# Install dependencies
pnpm install
 
# Create config file
vi config.js
 
# Execute locally
pnpm core start -c config.js --verbose --once

FAQ

Why this name?

The original intention of this project was to monitor updates of a Chinese VTuber group A-SOUL. Now this project has been renamed to eop for more general use.

Why not executing checks in parallel?

Most services have API limits or rate limits. Executing checks in parallel only make sense with small amount of accounts.

License

AGPL-3.0

On this page