Navigating to Yahoo News Comments Page

Yahoo News comments are on a dedicated /comments page, not the main article page.

URL Construction

Comments URL = Full Article URL + /comments

const fullArticleUrl = "https://news.yahoo.co.jp/articles/6564191";
const commentsUrl = `${fullArticleUrl}/comments`;
// Result: https://news.yahoo.co.jp/articles/6564191/comments

Navigation Code

try {
  const commentsUrl = `${fullArticleUrl}/comments`;

  await page.goto(commentsUrl, {
    waitUntil: 'networkidle2',
    timeout: 30000
  });

  // Wait for comments to load
  await new Promise(resolve => setTimeout(resolve, 2000));

  // Extract comments...
} catch (error) {
  console.log(`Failed to fetch comments: ${error.message}`);
  // Comments are not critical - continue without them
}

Error Handling

Comment extraction failures should not block article processing:

try {
  // Navigate and extract comments
} catch (error) {
  console.log(`Failed to fetch comments: ${error.message}`);
  // Continue without comments - not critical
}

Page Title

The comments page title contains total comment count:

(123)件のコメント - Yahoo!ニュース

Use regex to extract: /(\d+)件のコメント/