Navigating from Pickup Page to Full Article

Yahoo News articles require a two-step navigation: pickup page → full article page.

Navigation Flow

1. Start with pickup URL (e.g., https://news.yahoo.co.jp/pickup/6564191)
2. Find "記事全文を読む" link on pickup page
3. Navigate to full article URL (contains /articles/)

Code Example

// First, navigate to pickup page
await page.goto(pickupUrl, {
  waitUntil: 'networkidle2',
  timeout: 30000
});

// Wait for content to load
await new Promise(resolve => setTimeout(resolve, 2000));

// Find and extract full article URL
const fullArticleUrl = await page.evaluate(() => {
  const fullArticleLink = document.querySelector('a[href*="/articles/"]');
  return fullArticleLink ? fullArticleLink.href : null;
});

if (!fullArticleUrl) {
  throw new Error('Could not find full article link');
}

// Navigate to full article page
await page.goto(fullArticleUrl, {
  waitUntil: 'networkidle2',
  timeout: 30000
});

Link Selector

The full article link can be found using:

• a[href*="/articles/"] - Most reliable
• Text content: "記事全文を読む" (Read full article)

Error Handling

if (!fullArticleUrl) {
  throw new Error('Could not find full article link on pickup page');
}

Always check if the link exists before navigating.