x- 扫描器

以最快且最便宜的方式刮去X (Twitter)的微博. 包括企业团队在内的数以万计的客户使用经过战斗测试的基础设施.

环绕 认证 Twitter Scraper Lite 演员有开发者友好,以阶级为基础的API. 每一种方法都创造了一个在引擎盖下运行的辅助演员.

先决条件

• 节点.js 16+
• 一个 说明已付款计划的账户 —— 说 在这儿报名
• 一个 认证 API 令牌 —— 说 拿过来

安装

npm install @apidojo/x-scraper

CLI 使用量

从命令行安装到全球使用 :

npm install -g @apidojo/x-scraper

设置

# Store your Apify token (saved to ~/.x-scraper/config.json)
x-scraper init

# Or pass it per-command
x-scraper profile NASA --token apify_api_xxx

# Or set as environment variable
export APIFY_TOKEN=apify_api_xxx

图标

# Fetch tweets from a profile
x-scraper profile NASA --sort Latest --max-items 50

# Search by keyword
x-scraper keyword "artificial intelligence" --lang en --sort Latest

# Search by multiple keywords (joined with OR)
x-scraper keywords bitcoin ethereum solana

# Search by hashtag
x-scraper hashtag AI MachineLearning --sort Latest

# Fetch a single tweet by URL
x-scraper url https://x.com/elonmusk/status/1728108619189874825

# Fetch from multiple URLs
x-scraper urls https://x.com/a/status/1 https://x.com/b/status/2

# Tweets from a handle within a date range
x-scraper profile-date-range NASA --since 2024-01-01 --until 2024-06-01

# Tweets near a location
x-scraper location coffee --location "San Francisco" --radius 10mi

# Tweets with minimum engagement
x-scraper engagement bitcoin --min-likes 1000 --min-retweets 100

# Multiple profiles in one run
x-scraper profiles elonmusk naval paulg

# Mentions, media filters, cashtags, verified users, and more
x-scraper mention NASA
x-scraper media NASA
x-scraper images NASA
x-scraper videos NASA
x-scraper cashtag BTC ETH SOL
x-scraper verified cryptocurrency
x-scraper no-retweets elonmusk
x-scraper links TechCrunch
x-scraper conversation 1728108619189874825

# Raw actor input (full control)
x-scraper search --input '{"searchTerms":["from:NASA filter:media"]}'
x-scraper execute --input '{"searchTerms":["test"],"maxItems":10}'

输出选项

# Pretty JSON (default)
x-scraper profile NASA

# Compact JSON (pipe-friendly)
x-scraper profile NASA --json

# JSONL (one object per line)
x-scraper profile NASA --jsonl

# Save to file
x-scraper profile NASA --output tweets.json

# Debug mode (shows internal logs)
x-scraper profile NASA --debug

全球旗帜

快速启动( 图书馆)

// ESM
import { XScraper } from '@apidojo/x-scraper';

// CommonJS
const { XScraper } = require('@apidojo/x-scraper');

const scraper = new XScraper({ token: 'apify_api_xxxxxxxxxxxx' });

// Fetch latest tweets from NASA
const tweets = await scraper.getTweetsByProfile('NASA', { sort: 'Latest', maxItems: 50 });
console.log(tweets);

核心方法

execute(input, options?)

Raw通过——直接传递任何有效的演员输入对象. 当你知道演员期望的输入时使用这个.

const tweets = await scraper.execute({
  searchTerms: ['from:NASA filter:media'],
  sort: 'Latest',
  maxItems: 100,
});

search(input, options?)

同步运行刮取器——等待运行完成并返回所有项目. 选项 sort, (中文(简体) ), maxItems,以及 lang 自动合并到输入中.

const tweets = await scraper.search(
  { searchTerms: ['from:NASA'] },
  { sort: 'Latest', maxItems: 50, lang: 'en' }
);

searchAsync(input, options?)

启动跑步而不等待。 回返a RunHandle 投票和获得结果。 用于超过同步超时的大型运行 .

const run = await scraper.searchAsync(
  { searchTerms: ['from:NASA'] },
  { sort: 'Latest' }
);

await run.waitForFinish();
const tweets = await run.getItems();

stream(input, options?)

运行刮取器和流结果 作为同步执行器。 物品逐一取出,并有自动涂层.

for await (const tweet of scraper.stream({ searchTerms: ['from:NASA'] })) {
  console.log(tweet.text);
}

方便方法

所有方便方法都接受 options 对象 :

简介

// Tweets from a single profile
const tweets = await scraper.getTweetsByProfile('NASA', { sort: 'Latest' });

// Tweets from multiple profiles in one run
const tweets = await scraper.getTweetsByMultipleProfiles(
  ['elonmusk', 'naval', 'paulg'],
  { sort: 'Latest' }
);

// Tweets from a handle within a date range
const tweets = await scraper.getTweetsByHandleInDateRange(
  'NASA', '2024-01-01', '2024-06-01',
  { sort: 'Latest' }
);

// Exclude retweets from a handle
const tweets = await scraper.getTweetsExcludingRetweetsByHandle('elonmusk', {
  sort: 'Latest',
});

搜索

// Search by keyword
const tweets = await scraper.getTweetsByKeyword('artificial intelligence', {
  lang: 'en',
  sort: 'Latest',
});

// searchTweets is an alias for getTweetsByKeyword
const tweets = await scraper.searchTweets('artificial intelligence');

// Search by multiple keywords (joined with OR)
const tweets = await scraper.getTweetsByKeywords(
  ['bitcoin', 'ethereum', 'solana'],
  { lang: 'en', sort: 'Latest' }
);

// searchTweetsByMultipleKeywords is an alias for getTweetsByKeywords
const tweets = await scraper.searchTweetsByMultipleKeywords(['AI', 'ML', 'deep learning']);

// Search by hashtag
const tweets = await scraper.getTweetsByHashtag(['AI', 'MachineLearning'], {
  sort: 'Latest',
});

// Search by cashtag
const tweets = await scraper.getTweetsByCashtag(['BTC', 'ETH', 'SOL'], {
  lang: 'en',
  sort: 'Latest',
});

单调和 URL

// Fetch a single tweet by URL
const tweets = await scraper.getTweetByUrl(
  'https://x.com/elonmusk/status/1728108619189874825'
);

// Fetch from multiple URLs (tweets, lists, profiles)
const tweets = await scraper.getTweetsByUrls([
  'https://x.com/elonmusk/status/1728108619189874825',
  'https://twitter.com/i/lists/1234567890',
]);

对话

// Fetch replies in a conversation thread
const replies = await scraper.getTweetsByConversationId('1728108619189874825', {
  sort: 'Latest',
});

提及

// Tweets mentioning a user
const tweets = await scraper.getTweetsByMention('NASA', { sort: 'Latest' });

媒体过滤器

// Tweets with any media (images or videos)
const tweets = await scraper.getTweetsWithMediaByHandle('NASA');

// Tweets with images only
const tweets = await scraper.getTweetsWithImagesByHandle('NASA');

// Tweets with videos only
const tweets = await scraper.getTweetsWithVideosByHandle('NASA');

// Tweets with links
const tweets = await scraper.getTweetsWithLinksByHandle('TechCrunch');

接触过滤器( F)

// Tweets with minimum engagement
const tweets = await scraper.getTweetsWithMinEngagement('bitcoin', {
  minLikes: 1000,
  minRetweets: 100,
  sort: 'Top',
});

// Tweets from verified users only
const tweets = await scraper.getTweetsByVerifiedUsers('cryptocurrency', {
  sort: 'Top',
});

地点

// Tweets near a location
const tweets = await scraper.getTweetsByLocation('coffee', 'San Francisco', '10mi', {
  sort: 'Latest',
});

完全响应模式

默认情况下,方法只返回项目数组。 过 { fullResponse: true } 以获取运行的元数据 :

const result = await scraper.getTweetsByProfile('NASA', {
  sort: 'Latest',
  fullResponse: true,
});

console.log(result.items);     // Array of tweet objects
console.log(result.runId);     // Apify run ID
console.log(result.datasetId); // Apify dataset ID

与 RunHandle 同步运行

对于需要超过同步超时的大型运行:

const run = await scraper.searchAsync({ searchTerms: ['from:NASA'], sort: 'Latest' });

// Poll until the run finishes
await run.waitForFinish();

// Get all items
const tweets = await run.getItems();

// Or stream items
for await (const tweet of run.stream()) {
  console.log(tweet.text);
}

// Access run metadata
console.log(run.runId);
console.log(run.datasetId);
console.log(run.status);

处理错误

所有来自 Apify 的错误都直接抛出 。 盘点尝试/捕获的呼叫:

try {
  const tweets = await scraper.getTweetsByProfile('NASA');
} catch (error) {
  // Apify errors: 401 unauthorized, 404 not found, 429 rate limit, run failures
  console.error('Scraper error:', error.message);
}

如果没有提供令牌,则构造者立即投出:

try {
  const scraper = new XScraper({});
} catch (error) {
  // "APIFY_TOKEN is required. Pass it as: new XScraper({ token: "apify_api_xxx" })"
}

图文对象形状

每件退回的物品都是个微博对象:

{
  "type": "tweet",
  "id": "1728108619189874825",
  "url": "https://x.com/elonmusk/status/1728108619189874825",
  "text": "More than 10 per human on average",
  "retweetCount": 11311,
  "replyCount": 6526,
  "likeCount": 104121,
  "quoteCount": 2915,
  "createdAt": "Fri Nov 24 17:49:36 +0000 2023",
  "lang": "en",
  "isReply": false,
  "isRetweet": false,
  "isQuote": true,
  "author": {
    "userName": "elonmusk",
    "name": "Elon Musk",
    "id": "44196397",
    "followers": 172669889,
    "isVerified": true,
    "isBlueVerified": true
  }
}

API 参考

构造器

核心方法

方便方法

运行汉字

定价

此软件包使用 认证 Twitter Scraper Lite 使用事件定价的演员。 你只付你用过的 见 演员定价页面 现行费率.

许可证

麻省理工学院