Get news yohoo in python

PHOTO

Wed Sep 11 2024 20:06:12 GMT+0000 (Coordinated Universal Time)

from bs4 import BeautifulSoup
import requests
import json

# Initialize the list to store API data
api = []

# Base URL for Yahoo News
base_url = 'https://www.yahoo.com'

# Fetch the main page
url = f'{base_url}/news/'
response = requests.get(url)

# Check if the request was successful
if response.status_code == 200:
    soup = BeautifulSoup(response.text, 'html.parser')
    
    # Find all news items
    for news_item in soup.find_all('ul', class_='stream-items'):
        for item in news_item.find_all('li', class_='stream-item'):
            
            # Extract the article link
            item_id = item.find('a', class_='js-content-viewer')
            if item_id:
                link = item_id.get('href')
                full_link = f'{base_url}{link}'
                
                # Fetch the article page
                response2 = requests.get(full_link)
                if response2.status_code == 200:
                    soup2 = BeautifulSoup(response2.text, 'html.parser')
                    
                    # Extract article details
                    itemInfo = soup2.find('div', class_='caas-inner-body')
                    if itemInfo:
                        text1 = ''
                        for text in itemInfo.find_all('div', class_='caas-body'):
                            text1 += text.text

                        # Remove "View comments" from the text
                        text1 = text1.replace("View comments", "").strip()

                        # Extract additional details if available
                        image = item.find('img')['src'] if item.find('img') else ''
                        category = item.find('strong', class_='Tt(c)').text if item.find('strong', class_='Tt(c)') else ''
                        ell = item.find('span', class_='Ell').text if item.find('span', class_='Ell') else ''
                        title = item.find('h3', class_='stream-item-title').text if item.find('h3', class_='stream-item-title') else ''
                        description = item.find('p', class_='finance-ticker-fetch-success_D(n)').text if item.find('p', class_='finance-ticker-fetch-success_D(n)') else ''
                        
                        # Append data to the api list
                        api.append({
                            'link': full_link,
                            'image': image,
                            'category': category,
                            'ell': ell,
                            'title': title,
                            'description': description,
                            'text': text1,
                        })

# Convert the api list to a JSON-formatted string
api_json = json.dumps(api, indent=4)

# Write the JSON data to a file
with open('news_data.json', 'w') as file:
    file.write(api_json)

print("Data has been written to news_data.json")

COPY

Save snippets that work from anywhere online with our extensions

Comments

More like this

#html #javascript #nodejs

How to Save HTML Form Data in JSON - Express

<form action="/new" method="post">
 
  <input name="title" type="text">
  <input name="description" type="text">
  <button type="submit">Submit Form</button>
 
</form>

> More steps

#javascript

Get URL and URL Parts in JavaScript

var newURL = window.location.protocol + "//" + window.location.host + "/" + window.location.pathname + window.location.search

#javascript

Add event listener to multiple buttons with the same class

    btns = document.getElementsByClassName("saveBtn");
    for (var i = 0; i < btns.length; i++) {
        btns[i].addEventListener("click", function () {
			//Add function here
        });
    }

#javascript

Replace Textarea with Codemirror editor

<link rel="stylesheet" type="text/css" href="plugin/codemirror/lib/codemirror.css">

<body>
	<textarea class="codemirror-textarea"></textarea>
</body>

<script>

$(document).ready(function(){
    var codeText = $(".codemirror-textarea")[0];
    var editor = CodeMirror.fromTextArea(codeText, {
        lineNumbers : true
    });
});

</script>

<script type="text/javascript" src="plugin/codemirror/lib/codemirror.js"></script>

#javascript #nodejs #commandline

HTTPS Redirect in Nodejs

npm install heroku-ssl-redirect

> More steps

#javascript #nodejs

Save to MongoDB without Duplicates Using Mongoose

var mongoose = require('mongoose');
var Schema = mongoose.Schema;

const exampleSchema = new Schema({
    title: { type: String , required: true},
    content: [{type: String}]
});


var Example = mongoose.model('Example', exampleSchema);
module.exports = Example;

> More steps

#html #javascript #jsfunctions #dom #dommanipulation

getAttribute() - how to get the value of any attribute of HTML element

<html>

<input id="contact" name="address">

<script>

    var x = document.getElementById("contact").getAttribute('name');

</script>

</html>

#javascript

Full Stack Developer

function full_stack_developer() {
    full_stack_developer();
}

#javascript #javascript #functions #parameters

Required Parameters for Functions in JavaScript

 const isRequired = () => { throw new Error('param is required'); };

const hello = (name = isRequired()) => { console.log(`hello ${name}`) };

// These will throw errors
hello();
hello(undefined);

// These will not
hello(null);
hello('David');
The idea here is that it uses default parameters, like how the b parameter here has a default if you don’t send it anything:
function multiply(a, b = 1) {
  return a * b;
}

#javascript

3 Ways To Detect Selenium Bots Using Javascript - CodingTutz

if (navigator.webdriver) {
    document.body.innerHTML = "This is a Bot";
}

#javascript

Copy Text to Clipboard with Line Breaks

function copyToClipboard(){

    var codeToBeCopied = document.getElementById('code-snippet').innerText;
    var emptyArea = document.createElement('TEXTAREA');
    emptyArea.innerHTML = codeToBeCopied;
    const parentElement = document.getElementById('post-title');
    parentElement.appendChild(emptyArea);

    emptyArea.select();
    document.execCommand('copy');

    parentElement.removeChild(emptyArea);
    M.toast({html: 'Code copied to clipboard'})

    }

#javascript #promises #howto

How to Get the Domain From A URL Using Javascript

var url = "http://scratch99.com/web-development/javascript/";
var urlParts = url.replace('http://','').replace('https://','').split(/[/?#]/);
var domain = urlParts[0];

#javascript

Mapping an Object

const object1 = {
  a: 'somestring',
  b: 42
};

for (let [key, value] of Object.entries(object1)) {
  console.log(`${key}: ${value}`);
}

#javascript #jquery

jQuery.post() - AJAX for post requests

$.ajax({
  type: "POST",
  url: url,
  data: data,
  success: success,
  dataType: dataType
});

#javascript

How to store fetch response in javascript variable - Stack Overflow

async function fun() {
  return fetch('https://jsonplaceholder.typicode.com/todos/1').then(res => res.json());
}

const data  = await fun();

Get news yohoo in python

Save snippets that work from anywhere online with our extensions

Comments

More like this

Browse more snippets >>

Embed code snippet