The provided text consists of a series of coding tutorials and projects focused on Python. The initial tutorials cover fundamental Python concepts, including Jupyter Notebooks, variables, data types, operators, and conditional statements. Later tutorials explore looping constructs, functions, data type conversions, and practical projects like building a BMI calculator. The final segment introduces web scraping using Beautiful Soup and Requests, culminating in a project to extract and structure data from a Wikipedia table into a Pandas DataFrame and CSV.

Python Fundamentals: A Study Guide

Quiz

What is Anaconda, and why is it useful for Python development?

Anaconda is an open-source distribution of Python and R, containing tools like Jupyter Notebooks. It simplifies package management and environment setup for Python projects.

Explain the purpose of a Jupyter Notebook cell, and how to execute code within it.

A Jupyter Notebook cell is a block where code or markdown text can be written and executed. Code is executed by pressing Shift+Enter, which runs the cell and moves to the next one.

Describe the difference between code cells and markdown cells in a Jupyter Notebook.

Code cells contain Python code that can be executed, while markdown cells contain formatted text for notes and explanations. Markdown cells use a simple markup language for formatting.

What is a variable in Python, and how do you assign a value to it?

A variable is a named storage location that holds a value. Values are assigned using the assignment operator (=), such as x = 10.

Explain why variable names are case-sensitive, and provide an example.

Python treats uppercase and lowercase letters differently in variable names. For example, myVar and myvar are distinct variables.

List three best practices for naming variables in Python.

Use descriptive names, follow snake_case (words separated by underscores), and avoid starting names with numbers.

What are the three main numeric data types in Python?

The three main numeric data types are integers (whole numbers), floats (decimal numbers), and complex numbers (numbers with a real and imaginary part).

Explain the difference between a list and a tuple in Python.

A list is mutable (changeable), while a tuple is immutable (cannot be changed after creation). Lists use square brackets, while tuples use parentheses.

Describe the purpose of comparison operators in Python, and give three examples.

Comparison operators compare two values and return a Boolean result (True or False). Examples: == (equal to), != (not equal to), > (greater than).

Explain the purpose of the if, elif, and else statements in Python.

if executes a block of code if a condition is true. elif checks additional conditions if the previous if or elif conditions are false. else executes a block of code if none of the preceding conditions are true.

Quiz Answer Key

Anaconda is an open-source distribution of Python and R, containing tools like Jupyter Notebooks. It simplifies package management and environment setup for Python projects.
A Jupyter Notebook cell is a block where code or markdown text can be written and executed. Code is executed by pressing Shift+Enter, which runs the cell and moves to the next one.
Code cells contain Python code that can be executed, while markdown cells contain formatted text for notes and explanations. Markdown cells use a simple markup language for formatting.
A variable is a named storage location that holds a value. Values are assigned using the assignment operator (=), such as x = 10.
Python treats uppercase and lowercase letters differently in variable names. For example, myVar and myvar are distinct variables.
Use descriptive names, follow snake_case (words separated by underscores), and avoid starting names with numbers.
The three main numeric data types are integers (whole numbers), floats (decimal numbers), and complex numbers (numbers with a real and imaginary part).
A list is mutable (changeable), while a tuple is immutable (cannot be changed after creation). Lists use square brackets, while tuples use parentheses.
Comparison operators compare two values and return a Boolean result (True or False). Examples: == (equal to), != (not equal to), > (greater than).
if executes a block of code if a condition is true. elif checks additional conditions if the previous if or elif conditions are false. else executes a block of code if none of the preceding conditions are true.

Essay Questions

Discuss the differences between for loops and while loops in Python. Provide examples of situations where each type of loop would be most appropriate.
Explain the concept of web scraping using Python. What libraries are commonly used for web scraping, and what are some ethical considerations involved in web scraping?
Describe the process of defining and calling functions in Python. Explain the purpose of function arguments and return values, and provide examples of how to use them effectively.
Explain the different data types in Python and provide examples of using them in variable assignments, and data manipulation.
Explain the difference between an arbitrary argument, an arbitrary keyword argument, and an ordinary argument, and what are the use cases for each one.

Glossary of Key Terms

Anaconda: An open-source distribution of Python and R used for data science and machine learning, simplifying package management.
Jupyter Notebook: An interactive web-based environment for creating and sharing documents containing live code, equations, visualizations, and explanatory text.
Cell (Jupyter): A block in a Jupyter Notebook where code or markdown can be entered and executed.
Markdown: A lightweight markup language used for formatting text in markdown cells.
Variable: A named storage location that holds a value in a program.
Data Type: The classification of a value, determining the operations that can be performed on it (e.g., integer, string, list).
Integer: A whole number (positive, negative, or zero).
Float: A number with a decimal point.
String: A sequence of characters.
List: An ordered, mutable collection of items.
Tuple: An ordered, immutable collection of items.
Set: An unordered collection of unique items.
Dictionary: A collection of key-value pairs.
Comparison Operator: Symbols used to compare two values (e.g., ==, !=, >, <).
Logical Operator: Symbols used to combine or modify Boolean expressions (e.g., and, or, not).
if Statement: A conditional statement that executes a block of code if a condition is true.
elif Statement: A conditional statement that checks an additional condition if the preceding if condition is false.
else Statement: A conditional statement that executes a block of code if none of the preceding if or elif conditions are true.
for Loop: A control flow statement that iterates over a sequence (e.g., list, tuple, string).
while Loop: A control flow statement that repeatedly executes a block of code as long as a condition is true.
Function: A reusable block of code that performs a specific task.
Argument: A value passed to a function when it is called.
Return Value: The value that a function sends back to the caller after it has finished executing.
Web Scraping: Extracting data from websites using automated software.
Beautiful Soup: A Python library for parsing HTML and XML documents, making it easier to extract data from web pages.
Request: The act of asking a URL for its information.
HTTP request: A request using the standard “Hypertext Transfer Protocol,” which is the foundation for data communication on the World Wide Web.
CSV file: A Comma Separated Value file, which allows data to be saved in a table-structured format.
Pandas data frame: A two-dimensional, size-mutable, potentially heterogeneous tabular data structure with labeled axes.
List comprehension: An elegant syntax for creating lists based on existing iterables. It provides a concise way to generate lists using a single line of code, making it efficient and readable.

Python Programming and Web Scraping Tutorial

Okay, I have reviewed the provided text and here’s a briefing document summarizing its main themes and important ideas:

Briefing Document: Python Basics and Web Scraping Tutorial

Overall Theme:

This document contains the script for a video tutorial aimed at teaching beginners the fundamentals of Python programming, including setting up the environment, covering core concepts, and introducing web scraping with Beautiful Soup. The tutorial is structured as a hands-on lesson, walking the viewer through installing Anaconda, using Jupyter Notebooks, understanding variables, data types, operators, control flow (if/else, for loops, while loops), functions, data type conversions, and finally applying these skills to a web scraping project.

Key Ideas and Facts:

Setting up the Python Environment:
The tutorial recommends installing Anaconda, describing it as “an open source distribution of python and our products. So within Anaconda is our jupyter notebooks as well as a lot of other things but we’re going to be using it for our Jupiter notebooks.”
It walks through the Anaconda installation process, emphasizing the importance of selecting the correct installer for the operating system (Windows, Mac, or Linux) and system architecture (32-bit or 64-bit).
Introduction to Jupyter Notebooks:
Jupyter Notebooks are the primary environment for writing and executing Python code in the tutorial. “Right here is where we’re going to be spending 99% of our time in future videos this is where we’re going to write all of our code.”
Notebooks are comprised of “cells” where code or markdown can be written.
Markdown is introduced as a way to add comments and organization to the notebook: “markdown is its own kind of you could say language but um it’s just a different way of writing especially within a notebook.”
Basic notebook operations are explained, including saving, renaming, inserting/deleting cells, copying/pasting cells, moving cells, running code, interrupting the kernel, and restarting the kernel.
Python Variables:
A variable is defined as “basically just a container for storing data values.”
Variables are dynamically typed in Python, meaning the data type is automatically assigned based on the value assigned to the variable.
Variables can be overwritten with new values.
Multiple variables can be assigned values simultaneously (e.g., x, y, z = 1, 2, 3).
Multiple variables can be assigned the same value (e.g., x = y = z = “hello”).
Lists, dictionaries, tuples, and sets can all be assigned to variables.
The tutorial covers naming conventions such as camel case, Pascal case, and snake case, recommending snake case for readability: “when I’m naming variables I usually write it in snake case because I just find it a lot easier to read because each word is broken up by this underscore”.
It also covers invalid naming practices and what symbols can be used within variable names.
Strings can be concatenated using the + operator, but you can’t directly concatenate strings and numbers within a variable assignment (you can in a print statement using commas).
Python Data Types:
The tutorial covers the main data types in Python.
Numeric: Integers, floats, and complex numbers.
Boolean: True or False values.
Sequence Types: Strings, lists, and tuples.
Set: Unordered collections of unique elements.
Dictionary: Key-value pairs.
Strings can be defined with single quotes, double quotes, or triple quotes (for multi-line strings). Strings are arrays of bytes representing Unicode characters.
Indexing in strings starts at zero and can use negative indices to access characters from the end of the string. Slicing also works.
Lists are mutable, meaning their elements can be changed after creation using their indexes. Lists are index just like a string is. “One of the best things about lists is you can have any data type within them.”
Tuples are immutable, and lists and tuples are very similar with that being one main exception. “typically people will use tupal for when data is never going to change”.
Sets only contain unique values and are unordered.
Dictionaries store key-value pairs, and values are accessed by their associated keys. Dictionaries are changeable. “within a data type we have something called a key value pair… we have a key that indicates what that value is attributed to”.
Operators:
Comparison Operators: Used to compare values (e.g., ==, !=, >, <, >=, <=).
Logical Operators: Used to combine or modify boolean expressions (and, or, not).
Membership Operators: Used to check if a value exists within a sequence (in, not in).
Control Flow:
If/Elif/Else Statements: Used to execute different blocks of code based on conditions. The tutorial mentions “You can have as many ill if statements as you want but you can only have one if statement and one else statement”. Nested if statements are also covered.
For Loops: Used to iterate over a sequence, the diagram in the text walks through this, ending in “exit the loop and the for loop would be over”.
While Loops: Used to repeatedly execute a block of code as long as a condition is true. Break statements, continue statements, and using else statements to create a “counter” are also covered.
Functions:
Functions are defined using the def keyword.
Arguments can be passed to functions.
The tutorial describes many types of arguments including custom arguments, multiple arguments, arbitrary arguments, and keyword arguments. Arbitrary arguments use *args, and arbitrary keyword arguments use **kwargs.
Data Type Conversion:
Functions like int(), str(), list(), tuple(), and set() are used to convert between data types. This is important because the tutorial also says “it cannot add both an integer and a string.” Converting a list to a set automatically removes duplicate elements.
BMI Calculator Project:
The tutorial walks through building a BMI calculator.
The program takes user input for weight (in pounds) and height (in inches).
The BMI is calculated using the formula: weight * 703 / (height * height).
The program then uses if/elif/else statements to categorize the BMI into categories like underweight, normal weight, overweight, obese, severely obese, and morbidly obese.
The program uses input() for user input, which is then converted to an integer.
Web Scraping Project:
Introduction to Web Scraping: Web scraping is the process of extracting data from websites.
Libraries Used:requests: Used to send HTTP requests to retrieve the HTML content of a webpage.
Beautiful Soup: Used to parse the HTML content and make it easier to navigate and extract data. “Beautiful Soup takes this messy HTML or XML and makes it into beautiful soup”.
pandas: Used for data manipulation and analysis, specifically creating a DataFrame to store the scraped data. “We can use Pandas and manipulate this”.

Steps in Web Scraping:Send a Request: Use the requests library to get the HTML content of the target webpage.
Parse the HTML: Use Beautiful Soup to parse the HTML content into a navigable data structure.
Locate Elements: Use Beautiful Soup’s methods (e.g., find(), find_all()) to locate the specific HTML elements containing the data you want to extract.
Extract Data: Extract the text or attributes from the located HTML elements.
Store Data: Store the extracted data in a structured format, such as a pandas DataFrame or a CSV file.

Key Beautiful Soup Methods:find(): Finds the first element that matches the specified criteria.
find_all(): Finds all elements that match the specified criteria.
HTML Element Attributes: The tutorial mentions the importance of HTML element attributes (e.g., class, href, id) for targeting specific elements with Beautiful Soup.
Targeting Elements with Classes: The .find() and .find_all() methods can be used to select elements based on their CSS classes.
Navigating HTML Structure: The tutorial demonstrated how to navigate the HTML structure to locate specific data elements, particularly focusing on table, tr (table row), and td (table data) tags.
Data Cleaning: The tutorial showed how to clean up the extracted data by stripping whitespace from the beginning and end of the strings.
Creating Pandas DataFrame: The scraped data is organized and stored into a pandas DataFrame.
Exporting Data to CSV: The tutorial shows how to export the data in a data frame to a CSV file.

Quotes:

“Right here is where we’re going to be spending 99% of our time in future videos this is where we’re going to write all of our code.”
“a variable is basically just a container for storing data values.”
“when I’m naming variables I usually write it in snake case because I just find it a lot easier to read because each word is broken up by this underscore”
“typically people will use tupal for when data is never going to change”
“Beautiful Soup takes this messy HTML or XML and makes it into beautiful soup”
“we can use Pandas and manipulate this”
“It cannot add both an integer and a string”

Overall Assessment:

The provided text outlines a comprehensive introductory Python tutorial suitable for individuals with little to no prior programming experience. It covers a wide range of essential concepts and techniques, culminating in practical projects that demonstrate how these skills can be applied. The step-by-step approach and clear explanations, supplemented by hands-on examples, should make it accessible and engaging for beginners. However, some familiarity with HTML is assumed for the web scraping portion.

Python Programming: Basics and Fundamentals

Python Basics & Setup

1. What is Anaconda and why is it recommended for Python beginners?

Anaconda is an open-source distribution of Python and R, containing tools like Jupyter Notebooks. It simplifies setting up a Python environment, especially for beginners, by providing pre-installed packages and tools, avoiding individual installations and configurations that can be complex.

2. What is a Jupyter Notebook and how do you use it to write and run Python code?

A Jupyter Notebook is an interactive environment where you can write and execute Python code, as well as include formatted text (Markdown), images, and other content. You create cells within the notebook, type code, and then run each cell individually by pressing Shift+Enter.

3. What are variables in Python and why are they useful?

Variables are containers for storing data values. They are useful because they allow you to assign a name to a value (like a number, string, or list) and then refer to that value throughout your code by using the variable name, without having to rewrite the value itself.

4. How does Python automatically determine the data type of a variable, and what are some common data types?

Python uses dynamic typing, meaning it automatically infers the data type of a variable based on the value assigned to it. Common data types include integers (whole numbers), floats (decimal numbers), strings (text), Booleans (True/False), lists, dictionaries, tuples, and sets.

Python Fundamentals & Usage

5. What are the key differences between lists, tuples, and sets in Python?

Lists: Ordered, mutable (changeable) collections of items. They allow duplicate values.
Tuples: Ordered, immutable (unchangeable) collections of items. They also allow duplicate values.
Sets: Unordered collections of unique items. Sets do not allow duplicate values.

6. What are comparison, logical, and membership operators in Python, and how are they used?

Comparison Operators: Used to compare values (e.g., == (equal), != (not equal), > (greater than), < (less than)). They return Boolean values (True or False).
Logical Operators: Used to combine or modify Boolean expressions (e.g., and, or, not).
Membership Operators: Used to test if a value is present in a sequence (e.g., in, not in).

7. Explain the purpose of if, elif, and else statements in Python, and how they control the flow of execution.

if, elif (else if), and else statements are used to create conditional blocks of code. The if statement checks a condition, and if it’s true, the code block under the if statement is executed. elif allows you to check additional conditions if the previous if or elif conditions were false. The else statement provides a default code block to execute if none of the if or elif conditions are true.

8. How do for and while loops work in Python, and what are the differences between them?

for Loops: Used to iterate over a sequence (like a list, tuple, or string) and execute a block of code for each item in the sequence.
while Loops: Used to repeatedly execute a block of code as long as a specified condition is true. The loop continues until the condition becomes false.

The main difference is that for loops are typically used when you know in advance how many times you want to iterate, while while loops are used when you want to repeat a block of code until a specific condition is no longer met.

Creating Jupyter Notebooks with Anaconda

To create a notebook in Anaconda using jupyter notebooks, these steps can be followed:

Download Anaconda, which is an open source distribution of Python and R products, from the Anaconda website. Make sure to select the correct installer for your operating system (Windows, Mac, or Linux). For Windows users, it’s important to check the system settings to determine if it’s a 32-bit or 64-bit system.
Install Anaconda by clicking ‘next’ on the installer window. Review the license agreement and click ‘I agree’. Choose the installation type, either for the current user only or for all users on the computer. Select the file path for the installation, ensuring there is enough disk space (approximately 3.5 GB).
In the advanced options, it is not recommended to add Anaconda to the path environment variable unless you are experienced with Python. It is safe to register Anaconda as the default Python version. Allow the installation process to complete.
After the installation is complete, search for and open Anaconda Navigator.
In Anaconda Navigator, launch jupyter Notebook. This will open a new tab in the default web browser. If this is the first time opening jupyter Notebook, the file directory may be blank.
In the jupyter Notebook interface, go to the ‘new’ drop down and select ‘Python 3 (ipykernel)’ to create a new notebook with a Python 3 kernel.
A new jupyter Notebook will open where code can be written. This is where code will be written in future tutorials.
In the notebook, there are cells where code can be typed. To run the code in a cell, press Shift + Enter.
Besides writing code, markdown can be used to add comments and organize the notebook. To use markdown, type a hashtag/pound sign (#) followed by the text.

After creating a jupyter Notebook, the title can be changed by clicking on the name at the top of the page. It is also possible to insert cells, delete cells, copy and paste cells and move cells up or down.

Python Code: Concepts and Web Scraping

Python code involves several key concepts, including variables, data types, operators, control flow (if statements, loops), functions, and web scraping.

Variables:

Are containers for storing data values, such as numbers or strings.
A value can be assigned to a variable using the equal sign (=), for example, x = 22 assigns the value 22 to the variable x.
The print() function displays the value of a variable. For example, print(x) would output 22 if x has been assigned the value of 22.
Python automatically assigns a data type to a variable based on the assigned value.
Variables can be overwritten with new values.
Variables are case-sensitive.
Multiple values can be assigned to multiple variables. For example, x, y, z = “chocolate”, “vanilla”, “rocky road” assigns “chocolate” to x, “vanilla” to y, and “rocky road” to z.
Multiple variables can be assigned to one value. For example, x = y = z = “root beer float” assigns “root beer float” to all three variables.
Variables can be used in arithmetic operations. For example, y = 3 + 2 assigns the value 5 to the variable y.
Variables can be combined within a print statement using the + operator for strings or commas to combine different data types.

Data Types:

Are classifications of the data that you are storing.
Numeric data types include integers, floats, and complex numbers.
Integers are whole numbers, either positive or negative.
Floats are decimal numbers.
Complex numbers are numbers with a real and imaginary part, where j represents the imaginary unit.
Booleans have two built-in values: True or False.
Sequence types include strings, lists, and tuples.
Strings are arrays of bytes representing Unicode characters and can be enclosed in single quotes, double quotes, or triple quotes. Triple quotes are used for multi-line strings. Strings can be indexed to access specific characters.
Lists store multiple values and are changeable (mutable). Lists are defined using square brackets []. Lists can contain different data types. Items can be added to the end of a list using .append(). Items in a list can be changed by referring to the index number. Lists can be nested.
Tuples are similar to lists but are immutable, meaning they cannot be modified after creation. Tuples are defined using parentheses ().
Sets are unordered collections of unique elements. Sets do not allow duplicate elements. Sets are defined using curly brackets {}.
Dictionaries store key-value pairs. Dictionaries are defined using curly brackets {}, with each key-value pair separated by a colon :. Dictionary values are accessed using the key. Dictionary items can be updated, and key-value pairs can be deleted.

Operators:

Comparison operators compare two values.
== (equal to)
!= (not equal to)
> (greater than)
< (less than)
>= (greater than or equal to)
<= (less than or equal to)
Logical operators combine conditional statements.
and (returns True if both statements are true)
or (returns True if one of the statements is true)
not (reverses the result, returns False if the result is true)
Membership operators test if a sequence is present in an object.
in (returns True if a sequence is present in the object)
not in (returns True if a sequence is not present in the object)

Control Flow:

If statements execute a block of code if a condition is true.
if condition: (body of code)
else: (body of code) – executes if the initial if condition is false
elif condition: (body of code) – checks an additional condition if the initial if condition is false
Nested if statements can be used for more complex logic.
For loops iterate over a sequence (list, tuple, string, etc.).
for variable in sequence: (body of code)
Nested for loops can be used to iterate over multiple sequences.
While loops execute a block of code as long as a condition is true.
while condition: (body of code)
break statement: stops the loop even if the while condition is true
continue statement: rejects all the remaining statements in the current iteration of the loop
else statement: runs a block of code when the condition is no longer true

Functions:

Are blocks of code that run when called.
Defined using the def keyword.
Arguments can be passed to functions.
Arbitrary arguments allow an unspecified number of arguments to be passed.
Keyword arguments allow arguments to be passed with a key-value assignment.
Arbitrary keyword arguments allow an unspecified number of keyword arguments to be passed.

Web Scraping:

Involves extracting data from websites using libraries like Beautiful Soup and requests.
The requests library is used to send HTTP requests to a website.
Beautiful Soup is used to parse HTML content.
find() and find_all() methods are used to locate specific HTML elements.

Python Variable Assignment: A Comprehensive Guide

Variable assignment in Python involves using variables as containers for storing data values. You can assign a value to a variable using the equal sign (=). For example, x = 22 assigns the value 22 to the variable x. You can display the value of a variable using the print() function, such as print(x).

Key aspects of variable assignment:

Data Type Assignment: Python automatically assigns a data type to a variable based on the assigned value. For example, assigning 22 to x makes it an integer.
Overwriting: Variables can be overwritten with new values.
y = “mint chocolate chip”
print(y) # Output: mint chocolate chip
y = “chocolate”
print(y) # Output: chocolate
Case Sensitivity: Variables are case-sensitive. Y and y are treated as different variables.
Y = “mint chocolate chip”
y = “chocolate”
print(Y) # Output: mint chocolate chip
print(y) # Output: chocolate
Multiple Assignments: Multiple values can be assigned to multiple variables. For example:
x, y, z = “chocolate”, “vanilla”, “rocky road”
print(x) # Output: chocolate
print(y) # Output: vanilla
print(z) # Output: rocky road
One Value to Multiple Variables: Multiple variables can be assigned the same value. For example:
x = y = z = “root beer float”
print(x) # Output: root beer float
print(y) # Output: root beer float
print(z) # Output: root beer float
Combining Variables in Print: Variables can be combined within a print statement using the + operator for strings or commas to combine different data types. However, it is important to note that you can only concatenate a string with another string, not with an integer, unless you are separating the values by a comma in the print statement.
x = “ice cream”
y = “is”
z = “my favorite”
print(x + ” ” + y + ” ” + z) # Output: ice cream is my favorite
x = 1
y = 2
z = 3
print(x, y, z) # Output: 1 2 3

It is allowable to assign lists, dictionaries, tuples, and sets to variables as well.

Python Data Types: Numeric, Boolean, Sequence, Set, and Dictionary

Data types are classifications of the data that is stored; they inform what operations can be performed on the data. The main data types within Python include numeric, sequence type, set, Boolean, and dictionary.

Numeric data types include integers, float, and complex numbers.

An integer is a whole number, whether positive or negative.
A float is a decimal number.
A complex number is used for imaginary numbers, with j as the imaginary number.

Boolean data types only have two built-in values: either true or false.

Sequence type data types include strings, lists, and tuples.

Strings are arrays of bytes representing Unicode characters. Strings can be in single quotes, double quotes, or triple quotes. Triple quotes are called multi-line. Strings can be indexed, with the index starting at zero.
Lists store multiple values and are changeable. A list is indexed just like a string. A bracket means that it will be a list. Lists can have any data type within them. The comma in a list denotes that the values are separate. Lists can be nested.
Tuples are quite similar to lists but the biggest difference is that a tuple is immutable, meaning that it cannot be modified or changed after it is created. Typically, tuples are used when data is never going to change.

A set is similar to a list and a tuple, but does not have any duplicate elements. The values within a set cannot be accessed using an index, because it does not have one.

A dictionary is different than the other data types because it has a key value pair.

Web Scraping with Beautiful Soup and Requests

Web scraping involves extracting data from websites using libraries like Beautiful Soup and requests.

Key points regarding web scraping:

Libraries:
Requests: Used to send HTTP requests to a website to retrieve its HTML content. The requests.get() function sends a GET request to the specified URL and returns a response object. A response of 200 indicates a successful request.
Beautiful Soup: Used to parse HTML content, making it easy to navigate and search for specific elements.
Setting Up:
Import the necessary libraries:
from bs4 import Beautiful Soup
import requests
Specify the URL of the website to scrape:
URL = ‘https://example.com’
Send a GET request to the URL and retrieve the page content:
page = requests.get(URL)
Create a Beautiful Soup object to parse the HTML content:
soup = Beautiful Soup(page.text, ‘html.parser’)
HTML Structure:
HTML (Hypertext Markup Language) is used to describe the structure of web pages.
HTML consists of elements defined by tags (e.g., <html>, <head>, <body>, <p>, <a>).
Tags can have attributes, such as class, id, and href.
Inspecting web pages using browser developer tools helps identify the relevant HTML elements for scraping.
Finding Elements:
find(): Locates the first occurrence of a specific HTML element.
find_all(): Locates all occurrences of a specific HTML element and returns them as a list.
Elements can be filtered by tag name, class, id, or other attributes.
soup.find_all(‘div’, class_=’container’)
Extracting Data:
.text: Extracts the text content from an HTML element.
.strip(): Removes leading and trailing whitespace from a string.
Workflow:

Import libraries: Import Beautiful Soup and requests.
Get the HTML: Use requests to fetch the HTML content from the URL.
Parse the HTML: Create a Beautiful Soup object to parse the HTML.
Find elements: Use find() or find_all() to locate the desired elements.
Extract data: Use .text to extract the text content from the elements.
Organize data: Store the extracted data in a structured format, such as a list or a Pandas DataFrame.

Pandas DataFrames and Exporting to CSV:
The extracted data can be organized into a Pandas DataFrame for further analysis and manipulation.
The DataFrame can be exported to a CSV file using df.to_csv(). To prevent the index from being included in the CSV, use index=False.

Learn Python in Under 3 Hours | Variables, For Loops, Web Scraping + Full Project

The Original Text

what’s going on everybody welcome back to another video today we’re going to be learning the basics of python in under 3 [Music] hours python is a fantastic skill to know how to do but I remember when I was first learning python it was a little bit intimidating it a little bit more difficult than I was used to when I had just known Excel and SQL and python seemed really really difficult but I’ve been using python for over 7even years now is a fantastic skill to know how to use so in this really long lesson we’re going to be walking through every you need to know in order to get started in Python I’ll be walking you through how to set up your environment to make sure that you can actually run your code and then we’ll be walking through all of the basics all the variables and for loops and while loops and even web scraping and we’ll even have a full project in this as well so we have a ton of things to cover and I hope it is really helpful without further Ado let’s jump onto my screen and get started all right so let’s get started by downloading anaconda anaconda is an open source distribution of python and our products So within Anaconda is our jupyter notebooks as well as a lot of other things but we’re going to be using it for our Jupiter notebooks so let’s go right down here and if I hit download it’s going to download for me because I’m on Windows but if you want additional installers if you’re running on Mac or Linux then you can get those all right here now if you are running on Windows just make sure to check your system to see if it’s a 32-bit or a 64 you can go into your about and your system settings to find that information I’m going to click on this 64-bit it’s going to pop up on my screen right here and I’m going to click save now it’s going to start downloading it it says it could take a little while but honestly it’s going to take probably about two to three minutes and then we’ll get going now that it’s done I’m just going to click on it and it’s going to pull up this window right here we are just going to click next because we want to install it this is our license agreement you can read through this if you would like I will not I’m just going to click I agree now we can select our installation type and you can either select it for just me or if you have multiple admin or users on one laptop you can do that as as well for me it’s just me so I’m going to use this one as it recommends now it’s going to show you where it’s installing it on your computer this is the actual file path it’s going to take about 3.5 gigs of space I have plenty of space but make sure you have enough space and then once you do you can come right over here to next and now we can do some Advanced options we can add Anaconda 3 to my path environment variable and when you’re using python you typically have a default path with whatever python IG or notebook that you’re using I use a lot of Visual Studio code so if I do this I’m worried it might mess something up so I am not going to do this it also says it doesn’t recommend it again messing with these paths is kind of something that you might want to do once you know more about python so I don’t really recommend you having this checked we can also register in AA 3 as my default python 3.9 you can do this one and I’m going to keep it this way just so I have the exact same settings as you do so let’s go ahead and click install install and now it is going to actually install this on your computer now once that’s complete we can hit next and now we’re going to hit next again and finally we’re going to hit finish but if you want to you can have this tutorial and this getting started with anonda I don’t want either of them because I don’t need them but if you would like to have those keep those checked and you can get those let’s click finish now let’s go down and we’re going to search for Anaconda and it’ll say Anaconda navigate and we’re going to click on that and it should open up for us so this is what you should be seeing on your screen this is the Anaconda Navigator and this is where that distribution of python and R is going to be so we have a lot of different options in here and some of them may look familiar we have things like Visual Studio code spider our studio and then right up here we have our Jupiter notebooks and this is what we’re going to be using throughout our tutorials so let’s go ahead and click on launch and this is what should kind of pop up on your screen now I’ve been using this a lot um so I have a ton of notebooks and files in here but if you are just now seeing this it might be completely blank or just have some you know default folders in here but this is where we’re going to open up a new jupyter notebook where we can write code and all the things that we’re going to be learning in future tutorials and you can use this area to save things and create folders and organize everything if you already have some notebooks from previous projects or something you can upload them here but what we’re going to do is go right to this new we’re going to click on the drop down and we’re going to open up a Python 3 kernel and so we’re going to open this up right here now right here is where we’re going to be spending 99% of our time in future videos this is where we’re going to write all of our code so right here is a cell and this is where we can type things so I can say print I can do the famous hello world and then I’ll run that by clicking shift enter and this is where all of our code is going to go these are called cells so each one of these are a cell and we have a ton of stuff up here and I’m going to get to that in just a second one thing I want to show you is that you don’t only have to write code here you can also do something called markdown and so markdown is its own kind of you could say language but um it’s just a different way of writing especially within a notebook so all we’re going to do is do this little hashtag and actually I think it’s a pound sign but I’m going to call it hashtag we’re going to do that and we’re going to say first notebook and then if I run that we have our first notebook and we can make little comments and little notes like that that don’t actually run any code they just kind of organize things for us and I’m going to do that in a lot of our future videos so just wanted to show you how to do that now let’s look right up here a lot of these things are pretty important uh one of the first things that’s really important is actually saving this so let’s say we wanted to change the title to I’m going to do a AA because I want it to be at the beginning um so I can show you this I’m do AAA new notebook and I’m going to rename it and then I’m going to save that so if I go right back over here you can see AAA new notebook that green means that it’s currently running and when I say running I mean right up here and if we wanted to we go ahead and shut that down which means it wouldn’t run the code anymore and then we’d have to run up a new cluster uh so let’s go ahead and do that I didn’t plan on doing that but let’s do it so we have no notebooks running and right here it says we have a dead kernel so this was our Python 3 kernel and now since I stopped it it’s no longer processing anything so let’s go ahead and say try restarting now and it says kernel is ready so it’s back up and running and we’re good to go the next thing is this button right here now this is an insert cell below so if I have a lot of code I know I’m going to be writing I can click a lot of that and I often do that because I just don’t like having to do that all the time so I make a bunch of cells just so I can use them you can also delete cells so say we have some code here we’ll say here and we have code here and then we have this empty cell right here we can just get rid of that by doing this cut selected cells we can also copy selected cells so if I hit copy selected cells then I can go right here and say paste selected cells and as you can see it pasted that exact same cell you can also move this up and down so I can actually take this one and say I wanted it in this location I can take this cell and move it up or I can move it down and that’s just an easy way to kind of organize it in instead of having to like copy this and moving it right down here and pasting it you can just take this cell and move it up which is really nice now earlier when I ran this code right here I hit shift enter you can also run and it’ll run the cell below so you can hit run and it works properly if you’re running a script and it’s taking forever and it’s not working properly at least it’s you don’t think it’s working properly you can stop that by doing this interrupt the kernel right here and anything you’re trying to do within this kernel if it’s just not working properly it’ll stop it you can restart it then you can try fixing your code you can also hit this button if you want to restart your kernel and this button if you want to restart the kernel and then rerun the entire notebook as we talked about just a second ago we have our code and our markdown code we’re not going to talk about either of these because we’re not going to use that throughout the entire series the next thing I want to show you is right up here if you open this file we can create a new notebook we can open an existing notebook we can copy it save it rename it all that good stuff we can also edit it so a lot of these things that we were talking about you can cut the cells and copy the cells using these shortcuts if you would like to we also go to view and you can toggle a lot of these things if you would like to which just means it’ll show it or not show it depending on what you want so if we toggle this toolbar it’ll take away the toolbar for us or if we go back and we toggle the toolbar we can bring it back we can also insert a few different things like inserting a cell above or a cell below so instead of saying This plus button you can just say A or B adding above or below we also have the cell in which we can run our cells or run all of them or all above or all below and then we have our kernels right here which we were talking about earlier where we can interrupt it and restart those there are widgets we’re not going to be looking at any widgets in this series but if it’s something you’re interested in you can definitely do that then we have help so if you are looking for some help on any of these things especially some of these references which are really nice you can use those and you can also edit your own keyboard shortcuts and now that we walked through all of that you now have anacon and jupyter notebooks installed on your computer in future videos this is where we’re going to be writing all of our python code so be sure to check those out so we can learn python together hello everybody today we’re going to be learning about variables in Python a variable is basically just a container for storing data values so you’ll take a value like a number or a string you can assign it to a variable and then the variable will carry and contain whatever you put into it so for example let’s go right over here we’re going to say x and this is going to be our variable we’re going to say is equal to now we can assign the value to it so let’s say I want to put 22 x is now equal to 22 so we won’t have to write out the number 22 in later scripts that we write we can just say x because X is equal to 22 it now contains that number so now we can hit enter and say print we do an open parentheses and we’ll say x now I’m going to hit shift enter and now it prints out that 22 because we are printing x and x is equal to 22 this is our value and this is our variable one really great thing about variables is that it assigns its own data type it’s going to automatically do this so we didn’t have to go and tell X that it’s an integer it just automatically knew that 22 is a number so we can check that by saying type and then open parenthesis and writing X and we’ll do shift enter again and this says that X is an integer type now we only assigned a integer to X let’s try assigning a string value or some text to a variable so we’ll say Y is equal to uh let’s say mint chocolate chip I’m feeling some ice cream today so we’ll say mint chocolate chip now if we print that again we’ll do print open parentheses Y and do shift enter it’ll print mint chocolate chip and if we look at the type we can see that the type is a string this time and not an integer now again we did not tell it that X was an integer and Y was a string it just automatically knew this let’s go up here really quickly we’re going to add several rows in here because we’re about to write a lot of different variables and really learn in- depth how to use variables the next thing to know about variables is that you can overwrite previous variables right now we have mint chocolate chip and that is assigned to the variable y so if I go down here I say print y I hit shift enter it’s going to print out mint chocolate chip but if I go right above it I say Y is equal to and let’s say chocolate if I print that out it’s now going to say chocolate whereas up here I’m reassigning it to Y it’s still going to say mint chocolate chip so if I come right down here and I copy this and I’m going to paste this right here initially it is going to assign y to Chocolate but then right here it will automat Ally overwrite y as mint chocolate chip and when we hit shift enter it’s going to show mint chocolate chip variables are also case sensitive so if I come up here and I say a capital Y this is a lowercase Y and this is a capital Y it is going to print out the correct one instead of mint chocolate chip and then if I go down here to the print and I type the capital Y it will give us the mint chocolate chip up till now we’ve only assigned one value to one variable but but we can actually assign multiple values to multiple variables so let’s do X comma y comma Z is equal to and now we can assign multiple values to all of those so we can say chocolate and then we’ll do a comma oops a comma then we can say vanilla and then we’ll do another comma and we’ll say rocky road now now this is going to assign chocolate to X vanilla to Y and Rocky Road to Z so what we can do is we’ll say print and we’ll go print print print and we’ll say X Y and Z so it prints out chocolate vanilla and rocky road and these are our three different values we can also assign multiple variables to one value and we can do this by saying X is equal to Y is equal to Z is equal to and we can put whatever we would like let’s do root beer float then we’ll come back up here we’ll copy this and let’s print off our X our Y and Z and they are all the exact same now so far we’ve really only looked at integers and strings but you can assign things like lists dictionaries tupal and sets all to variables as well so let’s go right down here so let’s create our very first list I’m going to say ice _ cream is equal to and that is our variable right there the ice uncore cream is our variable so now we’re going to do an Open Bracket like this and we’re going to come up here and copy all of these values and we’re going to stick it within our list so now within ice cream we have three string values chocolate vanilla and rocky road all within this list so what we can do is we can say x comma y comma Z is equal to to ice cream so now these three values chocolate vanilla and rocky road will be assigned to these three variables X Y and Z and we can copy this print up here and we’ll hit shift enter and now the X Y and Z all were assigned these values of chocolate vanilla and rocky road now something that we just did which is really important or something that you really need to consider is how you name your variables so right here we have ice cream now this to me is exactly how I usually write my variables but there are many different ways that you can write your variables so let’s take a look at that really quickly and let’s add just a few more because I have a feeling we’re going to go a little bit longer than what we have so there are a few best practices for naming variables first I’m going to show you kind of what a lot of people will do I’ll show you some good practices and I’m going to show you some bad practices as well that you should avoid doing the first thing that we’re going to look at is something called camel case and let’s say we want to name it test variable case oops case now if we have a test variable case the camel case is going to look like this we’ll have lowercase test and then we’ll have uppercase variable and uppercase case is equal to this is what this variable is going to look like and we can assign it a nilla swirl and this is what your camel case will look like it’s going to be lowercase and then all the rest of those uh compound words or however you want to say that these letters are going to be capitalized to kind of separate where the words end and begin let’s go right down here we’re going to copy this the next one is called Pascal case so Pascal case is going to look just a little bit different instead of the lower case at test it’s going to be a capital T in test so test variable case again this is a very similar way of writing it very similar to camel case but just a capital at the beginning now let’s look at the last one and this one is my personal favorite this one is going to be the snake case now this one is quite a bit different in the fact that you don’t use any capital letters and you separate everything using underscore so we’re going to write testore variable uncore case now typically let me have them all in there typically these are the best practices these are what you typically want to do but probably the best one to use is this snake case right here what a lot of people say is that it improves readability if you take a look at either the camel case or the Pascal case which you will see people do it’s not as easy to distinguish exactly what it says and the name of a variable is important because you can gain information from it if people name them appropriately so when I’m naming variables I usually write it in snake case because I just find it a lot easier to read because each word is broken up by this underscore so now let’s look at some good variable names these are all ones that you can use or could use so let’s do something like test VAR so test VAR is completely appropriate we can also do something like testore VAR oops underscore we could do underscore testore VAR you’ll see that often as well well people will start it with an underscore you can do test VAR capital T oops capital T capital V in test VAR or you could even do something like test VAR two now adding a number to your variable is not inherently a Bad Thing usually it’s semif frowned upon but there are definitely some use cases where you can use it but one thing that you cannot do is do something like putting the two at the front if you put the two at the front it no longer works it won’t run properly at all so we’re going to take that out so we can’t do that so I’m going to use this as an example of what you should not do you also can’t use a dash so something like test- var2 that doesn’t work either and you also can’t use something like a space or a comma or really any kind of symbol like a period or a backslash or equal sign none of those things will work work within your variable now another thing that you can do within your variable is use the plus sign so let’s assign this we’ll say x is equal to and we’ll do a string we’ll say ice cream is my favorite and then we’ll do a plus sign and we’ll say period now what this will do is it will literally add these two strings together so let’s do print and we’ll do X so now it says ice cream is my favorite one thing that we cannot do in a variable is we cannot add a string and a number or an integer so we can’t do ice cream as my favorite two if we try to do that it will give us this error right here so in this error it’s saying you can only concatenate a string not an integer to a string so only a string plus a string for this example you can also do and we’ll say x is equal to or we’ll say y we’ll say Y is equal to 3 + 2 and it should output 5 because you can also do an integer and an integer now so far we’ve only been outputting one variable in the print statement but you can actually add multiple variables within a print statement so let’s go right down here we’re going to say let’s give it some more right there so we’ll say x is equal to ice cream and we’ll say Y is equal to is and then the last one Z is equal to my favorite and we’ll do a period at the end now we can go to the bottom and we can say print x + y + C and when we enter that and when we run and when we run that we get ice cream is my favorite now we can actually add a space before is a space before my and when we hit shift enter it says ice cream is my favorite you can also do this exact same thing with numbers as well so we’ll say x = to 1 2 and what Z is equal to 3 so this should equal six now one thing that we tried to do was assign to one variable a string plus an integer and that did not work but what you can do is you can take something like this and you can say ice cream and we’ll get rid of this one and we’ll get rid of the Z Now say Plus is actually not going to work let’s try running this so again we can’t concatenate these but what we can do in the print statement is we can separate it by a comma so when we add this comma it should work properly let’s hit enter and it says ice cream 2 again this makes no sense but you are able to combine a string and an integer separating by a comma now this is the meat and potatoes of variables there are some other things as well but some of those things are a little bit more advanced and not something I wanted to cover in this tutorial although we may be looking at some of those things in future tutorials but this is definitely the basics what you really really need to know about variables hello everybody today we’re going to be talking about data types in Python data types are the classification of the data that you are storing these classifications tell you what operations can be performed on your data we’re going to be looking at the main data types within python including numeric sequence type set Boolean and dictionary so let’s get started actually writing some of this out and first let’s look at numeric there are three different types of numeric data types we have integers float and complex numbers let’s take a look at integers an integer is basically just a whole number whether it’s positive or negative so an integer could be a 12 and we can check that by saying type we’ll do an open parenthesis and a Clos parenthesis and if we say the type of 12 it’s going to give us an integer or if we say a -2 that is also an integer we can also perform basic calculations like -2 + 100 and that’ll tell us it is also an integer so whether it’s just a static value or you’re performing an operation on it it’s still going to be that data type if those numbers are whole numbers whether negative or positive now let’s take this exact one and let’s say 12 and we’ll do plus 10.25 when we run this it’s no longer going to be a whole number it’ll now be a float so let’s check this now this is a float type because is no longer a whole number it’s now a decimal number and the last data type within the numeric data type is called complex let’s copy this right down here now personally this is not one that I’ve used almost ever but it is one just worth noting so you can do 12 plus and let’s say 3 J and if we do this it’s going to give us a complex the complex data type is used for imaginary numbers for me it’s not often used but if you do use it J is used as that imaginary number if you use something like C or any other number it’s going to give you an error J is the only one that will work with it now let’s take a look at Boolean values so we’ll say Boolean the Boolean data type only has two built-in values either true or false so let’s go right down here and say type true and when we run this it’ll say bu which stands for Boolean we can do the exact same thing with false that is also Boolean and this can be used with something like a comparison operator so let’s say 1 is greater than 5 and let’s check this this is giving us a Boolean because it’s telling us whether one is greater than five let’s bring that right down here this will give us a false so it’s telling us that one is not greater than five and just as we got a false we can say 1 is equal to 1 and this should give us a true so now let’s take a look at our sequence type data types and that includes strings lists and tupal we let’s start off by looking at string strings in Python strings are arrays of byes representing Unicode characters when you’re using strings you put them either in a single quote a double quote or a triple quote I call them apostrophes it’s just what I was raised to call them but most people who use Python call them quotes So Right Here we have a single quote and that works well we can do a double quote and that works also and as you can see they are the exact same output and then we have a triple quote just like this and this is called a multi-line so we can write on multiple lines here so let’s write a nice little poem so we’ll say the ice cream vanquished my longing for sweets upon this diet I look away it no longer exists on this day and then if we run that it’s going to look a little bit weird it’s basically giving us the raw text which is completely fine but let’s let’s call this a multi-line and we’re going to call this a variable multi-line and we’re going to come down here and say print and before I run this I have to make sure that this is Ran So now let’s print out our multi-line and now we have our nice little poem right down here now something to know about these single and double quotes is how they’re actually used so if we use a single quote and we say I’ve always wanted to eat a gallon of ice cream and then we do an apostrophe at the end obviously something went wrong here what went wrong is when you use a single quote and then within your text within your sentence you have another apostrophe it’s going to give you an error so what we want to do is whenever we have a quote within it we need to use a double quote these double quotes will negate any single quotes that you have within your statement they won’t however negate another double quote so you need to make sure you aren’t using double quotes within your sentence if you want to do something like that you need to use the triple quotes like we did above so we can do double double and then let’s paste this within it and anything you do Within These triple quotes will be completely fine as long as you don’t do triple quotes within your triple quotes we’ll say this is wrong so even though it’s between these two triple quotes it doesn’t work exactly again you just have to understand how that works you have to use the proper apostrophes or quotes within your string and just to check this we can always say here’s our multi-line we can always say type of multi-line and that is still a string one really important thing to know about strings is that they can be indexed indexing means that you can search within it and that index starts at zero so let’s go ahead and create a variable and we’ll just say a is equal to and let’s let’s do the all poopular hello world let’s run this and now when we print the string we can say a and we’re going to do a bracket and now we can search throughout our string using the index so all you have to do is do a colon we going say five what this is going to do is is going to say zero position zero all the way up to five which should give us the whole hello I believe let’s run this and it’s giving us the first five positions of this string we can also get rid of the colon and just say something like five and then when we run this it’s actually going to give us position five so this is 0 1 2 3 4 and then five is the space let’s do six so we can see the actual letter and that is our w we can also use a negative when we’re indexing through our string so we could say -3 and it’ll give us the L because it’s -1 2 and three we can also specify a range if we don’t want to use the default of Z so before we did 0 to 5 and it started at zero because that was our default but we could also do 2 to 5 let’s run this and now we go position 0 1 and then we start at 2 L L O now we can also multiply strings and we have this a hello world so we can do a time three and if we run this it’ll give us hello world three times and we can also do A+ a and that is hello world hello world now let’s go down here and take a look at lists lists are really fantastic because they store multiple values the string was stored as one value multiple characters but a list can store multiple separate values so let’s create our very first list we’ll say list really quickly and then we’ll put a bracket and a bracket means this is going to be a list there are other ones like a squiggly bracket and a parentheses these denote that they are different types of data types the bracket is what makes a list a list so to keep it super simple we’ll say 1 2 3 and we’ll run this and now we have a list that has three separate values in it the comma in our list denotes that they are separate values and a list is indexed just like a string is indexed so position zero is this one position one is the two and position two is the three now when we made this list we didn’t have to use any quotes because these are numbers but if we wanted to create a list and we wanted to add string values we have to do it with our quotes so we’ll say quote cookie dough then we’ll do a comma to separate the value and then we’ll say strawberry and then we’ll do one more and this will just be chocolate and when we run this we have all three of these values stored in our list now one of the best things about list is you can have any data type within them they don’t just have to be numbers or strings you can basically put anything you want in there so let’s create a new list and let’s say vanilla and then we’ll do three and then we’ll add a list within a list and we’ll say Scoops comma spoon and then we’ll get out of that list and then we’ll add another value of true for Boolean and now we can hit shift enter and we just created a list with several different data types within one list now let’s take this one list right here with all of our different ice cream flavors we’ll say icore cream is equal to this list now one thing that’s really great about lists is that they are changeable that means we can change the data in here we can also add and remove items from the list after we’ve already created it so let’s go and take ice cream and we’ll say ice cream. append and this is going to append it to the very end of the list we do an open parenthesis let’s say salted caramel now when we run this and we call it just like this it’s going to take this list add salted caramel to the end and we’ll print it off and as you can see it was added to the list and just like I said before let me go down here we can also change things from this list so let’s say ice cream and then we need to look at the indexed position so we’re going to say zero and that’s going to be this cookie dough right here we can say that is equal to so we can now change that value so let’s call that butter peon and now when we call it we can now see that the cookie dough was changed to butter peon another thing that you saw just a little bit ago is something called a list within a list basically a nested list so we had Scoops spoon true let’s give this and we’ll say nested uncore list is equal to now when we run this we now have this nested list so if we look at the index and we say 0 we’ll get vanilla if we say two we’ll get scoops and spoons now since we have a list within a list we can also look at the index of that nested list so let’s now say one and that should give us just spoon and you can go on and on and on with this you can do lists within lists within lists and all of them will have indexing that you can call now let’s go down here and start taking a look at tupal so a list and a tupal are actually quite similar but the biggest difference between a list in a tupal is that a tupal is something called immutable it means it cannot be modified or changed after it’s created let’s go right up here we’re going to say Tuple and let’s write our very first tupal so we’ll say Tuple undor Scoops is equal to and then we’ll do an open parenthesis now these open parentheses you’ve seen if you do like a print statement but that’s different because that’s executing a function this is actually creating a tupal which is going to store data for us so we’ll say one 2 three two and one let’s go ahead and create that tupal and we can just check the data type really quickly and it’s a tupal and just like we saw before a tupal is also indexed so if we go at the very first position which is a one we will get the output of a one but we can’t do something like aend and then add a value like three if we do that it’s going to say tupal object has no attribute aend it’s just because you cannot change or add anything to a tupal just like we were talking about before typically people will use tupal for when data is never going to change an example for this might be something like a city name a country a location something that won’t change they definitely have their use cases but I don’t think they’re as popular as just using a list so now let’s scroll down and start taking a look at sets but really quickly let me add a few more cells for us and let’s say sets now a set is somewhat similar to a list and a tupal but they are little bit different in the fact that they don’t have any duplicate elements another big difference is that the values within a set cannot be accessed using an index because it doesn’t have an index because it’s actually unordered we can still Loop through the items in a set with something like a for Loop but we can’t access it using the bracket and then accessing its index point so let’s go ahead and create our very first first set so we’re going to say daily pints then we’re going to say equal to and to create a set we’re going to use these squiggly brackets I don’t know if there’s an actual name for those if I’m being honest I call them squiggly brackets and that’s what we’re going to go with we’re going to put in a one a two and a three so let’s go ahead and run this and let’s look at the type and as you can see it is a set now when we print this out it’s going to show us one a two and a three and those are all the values Within set but if we copy this and we’ll say daily pant log this is going to be every single day maybe I had different values now when we run this and we do the exact same thing now when we print this it’s going to have just the unique values within that set now a use case for set and this is something that I’ve done in the past is comparing two separate sets maybe you have a list or a tupal and you convert that into a set and that will narrow it down down to its unique values then you can compare the unique values of one set to the unique values in another set and then we can see what’s the same and what’s different so let’s go down here and let’s say wife’s uncore daily and we’ll just copy this right here we’ll say is equal to let’s do our squiggly lines let’s do one two let’s do just random numbers so now this is my daily log and this is my wife’s daily log and now we can compare these values so let’s go right down here let’s say print we’ll do my daily logs and then we’ll do this bar right here and this is going to show us the combined unique values it’s basically like putting them all in one set and then trimming it down to just the unique values so we’ll take wife’s daily pintes log and when we run this we actually need to run this first when we run this we should see all the unique values between these two sets and so as you can see 0 1 2 3 4 5 6 7 24 31 so these are all the unique values between these two sets we can also do another one and instead of this bar we’re going to do this symbol right here which I believe is called an Amper sand don’t quote me on that but when we run this it’s going to show what matches that means which ones show up in both sets so the only ones that show up in both sets are 1 2 3 and five we can also do the opposite of that by doing a minus sign and this is going to show us what doesn’t match and so we have 4 6 and 31 now where is our 24 that was in our wife’s daily pints log it’s in this one but we’re subtracting the values on this one so let’s reverse this and we’ll say daily pints log and let’s run it now those are our other values so we’re taking the values of this and then we’re subtracting all the ones that are the same and getting the remaining values and then for our last one we can get rid of this and we’ll do this symbol right here and this is going to show if a value is either in one or the other but not in both so let’s run this so these values are completely unique only to each of those sets now the very last one that we’re going to look at in this video is dictionaries so let’s go right down here let’s add a few cells and let’s say dictionaries now I saved dictionary for last because this one is probably the most different out of all the previous data types that we’ve looked at within a data type we have something called a key value pair that means when we use a dictionary it’s not like a list where you just have a value comma value comma value we have a key that indicates what that value is attributed to so let’s write out a dictionary to see how this looks so we’re going to say dictionary cream and just like a set we use a squiggly line but the thing that differentiates it is that in a dictionary we’ll have that key value pair whereas in a set each value is just separated by a comma so let’s write name and this is our key and then we do a colon and this is then where we input our value so we’re going to say Alex freeberg and then we separate that key value Pair by a comma and now we can do another key value pair so we’ll say weekly intake and and a colon and we’ll say five pints of ice cream do a comma and then we’ll do favorite ice creams and now what we’re going to do is we’re going to put in here a list so within this dictionary we can also add a list we’ll do MCC from mint chocolate chip and then we’ll add chocolate another one of my favorites so now we have our very first dictionary let’s copy this and run it and let’s just look at the type and as as you can see it says that this is a dictionary let’s also print it out now if we want to we can take our dictionary cream and say dot values with an open parenthesis and when we execute this we’ll see all of the values within this dictionary so here’s our values of Alex freeberg five mint chocolate chip and chocolate we can also say keys and when we run this all of the keys the name weekly intake and favorite ice creams and we can also say items so this key value pair is one item and this key value pair is another item now one difference between something like a list and a dictionary is how you call the index but you can’t call it by doing something like this where you just do a bracket oops and say zero so this would in theory take this very first one right our very first key value pair that’s going to give us an error how you call a dictionary is actually by the key so it doesn’t technically have an index but you can specify what you want to call and take it out so we’re going to say name and this is going to call that key right here and when we run this we’ll get the value which is Alex freeberg one other thing that you can do is you can also update information in a dictionary which we can’t with some other data types so for this for the name it was Alex freeberg now let’s say Steen freeberg and when we update that I’m also going to print the dictionary get rid of this so it’s going to update Christine freeberg in that value of the name so let’s go ahead and run this and now it changed the name from Alex freeberg to Christine freeberg we can also update all of these values at one time so let’s copy this and I’m going to put it right down here I’m going to say dictionary.c cream. update then we’re going to put a bracket or not a rocket but a parenthesis around these so now what we’re going to do is update this entire thing let me take this say print this dictionary now we can update this to anything we want so instead of here I can say I’ll say weight and because of all that ice cream I now weigh 300 lb so let’s run this and as you can see it did not delete our key value pair right here instead it just added to it when you’re using the update we can’t actually delete that’s the delete statement and I’ll show you that in just a second but all we did was added this new value it also is going to check and see if you changed anything with your key value pair so we can go in here and change this value and we’ll say 10 so now when we run this the value of this key value pair was changed but let’s say we do want to delete it we’ll say deel that stands for delete part of this dictionary cream and now let’s specify the key which will also delete the value with it but let’s specify the key that we want to get rid of and let’s say wait and then let’s print that again and as you can see the weight was deleted from that dictionary so hello everybody today we’re going to be taking a look at comparison logical and membership operators in Python operators are used to perform operations on variables and values for example you’re often going to want to compare two separate values to see if they are the same or if they’re different within Python and that’s where the comparison operator comes in right here you can see our operators you can also see what they do so this equal sign equal sign stands for equal we have the does not equal the greater than less than greater than or equal to and less than or equal to and honestly I use these almost every single time I use Python so these are very important to know and know how to use so let’s get rid of that really quickly and actually start writing it out and see how these comparison operators work in Python the very first one that we’re going to look at is equal to now you can’t just say 10 is equal to 10 let’s try running that really quickly by clicking shift enter it’s going to say cannot assign to literal that’s because this is like assigning a variable we’re trying to say 10 is equal to 10 and then we can call that 10 later but that’s not how this actually works what we’re trying to do is to determine whether 10 is equal to 10 so we’re going to say equal sign equal sign and then if we run that by clicking shift enter again it’s going to say true now if we put something else like 50 in there and we try to run this it’s going to say false so really what you’re going to get when you use these comparison operators is either a true or a false if we take this right down here we can also say does not equal and we’re going to use an exclamation point equal sign and that says 10 is not equal to 50 and that should be true you can also compare strings and variables so let’s go right down here and we’re going to say vanilla is not equal to chocolate and when we run this it’ll say false now if it was the same just just like when we did our numbers it should say true and we can also compare variables so we’ll say x is equal to vanilla and Y is equal to chocolate and then when we come down here we can say x is equal to Y and it’ll give us a false and we say X is not equal to Y and it’ll give us a true the next one that we’re going to take a look at is the less than so let’s copy this one right up here let’s scroll down and let’s say 10 is less than 50 now this will come out as true now let’s say we put a 10 in here before 10 was of course less than 50 but is 10 less than 10 no that’s false because they are the same so if we want an output that is true all we would have to add is an equal sign right here and this would say 10 is less than or it is equal to 10 and now it’s true of course we can say the exact same thing by saying greater than so 10 is equal or greater than 10 that’ll be true because 10 is equal to 10 we can also say 50 is greater or equal to 10 because 50 is obviously greater than 10 now let’s look at logical operators that are often combined with comparison operators so our operators are and or and not so if you have an and that returns true if both statements are true if it’s or only one of the statements has to be true and the not basically reverses the result so if it was going to return true it would return turn false I don’t use this not one a lot but I will show you how it works so let’s actually test that out so before we were saying 10 is greater than 50 and of course this returned false so now let’s add a parentheses around this 10 is greater than 50 and we’re going to say and we’ll do an open parenthesis 50 is greater than 10 now this statement right here is true 50 is greater than 10 so we have a true statement and a false statement but this and is going to look at both of them it’s going to say they both need to be true in order to return a true so let’s try running this and we still have a false if we want it to return true we’re going to have to change this to make it a true statement so 70 is greater than 50 and 50 is greater than 10 when we run this it should return true now let’s look at the or so let’s copy this and we’ll say 10 is greater than 50 or 50 is greater than 10 now this is a false statement and this is a true statement so if even one of them is a true statement the output should be true and again we can do this even with strings so we can do vanilla and chocolate there we go and vanilla is actually greater than chocolate because V is a higher number in the alphabetical order so V is like 20 something whereas chocolate is three right so it actually looks at the spelling for this so if we say or here it will come out true and if we say and here it should also be true because V is greater than C and 50 is greater than 10 so this should also be true now let’s copy this right here and we’re going to say not so what we had before is 50 is greater than 10 that returned true but now all we’re doing is putting not in front of it so instead of returning true it’s going to return false so now let’s take a look at membership operators and we use this to check if something whether it’s a value or a string or something like that is within another value or string or sequence our operators are in and not in so it’s pretty simple if it’s in it’s going to return true if the sequence with a specified value is present in the object just like we were talking about and for not in it’s basically the exact same thing if it’s not in that object so let’s start out by taking a look at a string we’re going to say icore cream is equal to I love chocolate ice cream and then we’re going to say love in icore cream and that will will turn true so all we’re doing is searching if the word love or that string is in this larger string we could also just do that by literally copying this and putting this where this is so we can check is this string part of this string and it’ll say true we can also make a list so we’ll say Scoops is equal to and then we’ll do a bracket and we’ll say 1 2 3 4 5 and then we’ll say two in Scoops so all we’re doing is searching to see if two is within this list and that should return true now if we put a six here and we said not in it will also return true because six is not in scoops and that is true and just like we did we could also say wanted underscore Scoops and we’ll say eight so I wanted eight Scoops so we can say wanted Scoops in Scoops and this should return true because there’s not an eight within the Scoops that we wanted and if we said in and we said we wanted eight is that within our list that we created and that’s going to return a false hello everybody today we’re going to be taking a look at the if statement within python now it’s actually the if lfl statement but that’s a mouthful so I’m just going to call it the if L statement now we have this flowchart and I apologize for being blurry but this is the absolute best one that I could find right up top we have our if condition now if this if condition is true we’re going to run a body of code but if that condition is false we’re going to go over here and go to the LF condition the LF condition or statement is basically saying if the first if statement doesn’t work let’s try this if statement if this LF statement is true it goes to this body of code if it’s false it’ll come over here to the else and the else is basically if all these things don’t work then run this body of code now you can have as many ill if statements as you want but you can only have one if statement and one else statement so let’s write out some code and see how this actually looks let’s first start off by writing if that is our if statement and now we have to write our condition which is about to be either met or not met so we’ll say if 25 is greater than 10 which is true we’ll say colon and then we’re going to hit enter and it’s going to automatically indent that line of code for us and this is our body of code so if 25 is greater than 10 our body of code will execute so for us we’re just going to write print and we’ll say it worked now if we run this it’s going to check is 25 greater than 10 if that is true print this so let’s hit shift enter and it worked now let’s take this exact code we’ll paste it right down here and we’ll say is less than and right now this if statement is not true so it’s not actually going to work as you can see there’s no output there’s nothing that happened really but it did check to see if 25 was less than 10 but it just wasn’t true now we can use our else statement so we’re going to come right down here and we’re going to say else and we’ll do a colon and we’ll hit enter again automatically indenting and we’re going to say print and we’re going to say it did not work dot dot dot so what it’s going to do is it’s going to come up here and check is 25 less than 10 no it’s not so this body of code is not going to be executed it’s going to go right down to this else statement now this else statement is going to be printed there’s no condition on this so the if statement has a condition 25 is less than 10 this has no condition so if this doesn’t work if this is false it’s going to come down here and it will run this body of code let’s run this by clicking shift enter and as you can see our output is it did not work now let’s go back up here and put greater than because this is now true it’s going to say if 25 is greater than 10 print it worked and then it’s going to stop it’s not going to go to this L statement at all so let’s run this and our output is it worked so what if we have a lot of different conditions that we want to try let’s come right down here this is where the LF comes in so really quickly let’s change this to a not true a false statement we’re going to go down and say LF and we’re going to say if it is and let’s say 30 we’ll say LF worked so now it’s going to check is 25 less than 10 no it’s not let’s look at the next condition is 25 less than 30 and if it is we’ll print L if worked so let’s try running this and L if worked now we can do as as many of these LF statements as we want we can do let’s just try a few of them right here so we’ll say if 25 is less than 20 is less than 21 and let’s do 40 and let’s do 50 so we’ll say LF lf2 lf3 and lf4 now if you look at this the first one that is actually going to work is this 25 to 40 right here once this one is checked and it comes out as true none of the other LF or L statements will work so let’s try this one it should be lf3 and this one ran properly now within our condition so far we’ve only used a comparison operator we can also use a logical operator like and or or so we can say if 25 is less than 10 which it’s not let’s say or actually and we’ll say or 1 is less than three which is true if we run this now it will actually work so we can use several different types of operators within our if statement to see if a condition is true or not or several conditions are true there’s also a way to write an if else statement in one line if you want to do that so we can write print we’ll say it worked and then we’ll come over here and say if 10 is greater than 30 and then we’ll write else print and we’ll say it did not work just like we had before except now it’s all occurring on one line so let’s just try this and see if it works so it’s saying print it worked if 10 is greater than 30 which it wasn’t so it went to the L statement and then it printed out our body right here although we didn’t have any indentation or multiple lines it was all done in one line now there’s one other thing that we haven’t looked at yet uh and I’m going to show it to you really quickly and that’s a nested if statement so when we run this it’s going to say it worked it works because it says 25 is less than 10 or 1 is less L than three since this is true it’s going to print out it worked but we can also do a nested if statement so we can do multiple if statements as well so we’re going to hit enter and we’ll say if and we’ll do a true statement here so we’ll say if 10 is greater than 5 let’s do a colon hit enter then we’ll say print and then we’ll type A String saying this nested if statement oops worked now let’s try this out and see what we get so it went through the first if statement it said it was true and it prints out it worked this is still the body of code so it goes down to this next if statement and it says if 10 is greater than five we’re going to print this out and you could do this on and on and on it can basically go on forever and you can create a really in-depth logic and that actually happens a lot when you start writing more advanced code hello everybody today we’re going to be learning about four Loops in Python the for Loop is used to iterate over a sequence which could be a list a tube an array a string or even a dictionary here’s the list that we’ll be working with throughout this video and I have this little diagram right here which kind of explains how a for Loop works the for Loop is going to start by looking at the very first item in our sequence or our list and that’s going to be our one right here it’s going to ask is this the last element in our list and it is not so it’s going to go down to this body of the for Loop now we can have a thousand different things that can happen in the body of the for loop as we’re about to look out in just second then it’s going to go up to the next element and ask is this the last element reached so it’ll be no again because we’ll be going to the two and then the three and then the four and the five once it reaches the five it’ll go to the body of the for Loop and then when it asks if that’s the last element the answer would be yes because it’s iterated through all the items within the list and then we would exit the loop and the for Loop would be over now that may not have made perfect sense but let’s actually start writing out the syntax of a for Loop so we can understand understand this better to start our for loop we’re going to say four and then we’re going to give it a temporary variable for this for Loop so it’s a variable as it iterates through these numbers it’s going to assign the variable to that number so for this one we’re just going to say number because it’s pretty appropriate because these are all numbers and then we’re going to say in integers now right here you can put just about anything this could be the list this could be a tuple this could be a string even but that is what we’re going to iterate through so we’re saying for the variables each of these numbers within this list of integers and then we’re going to write a colon this is the body of code that’s going to actually be executed when we run through and iterate through our list so for our first example we’re going to start off super simple and all we’re going to do is say print open parentheses and say number as it iterates through the one two 3 4 and five number becomes our variable that is going to be printed so during that first loop our one will be printed because that will be assigned right here then through the next iteration the two will be assigned and it’ll be put right here in each Loop until the very end so let’s hit shift enter and as you can see it did exactly that now in this body and I’ll copy and paste this down here in this body we really can do just about anything we want we don’t even have to use this variable number right here we can just print yep if we wanted to and and what it’s going to do is for each iteration all five of those every time it Loops through it’s going to print off yep so let’s hit shift enter and it printed it off for us so really we weren’t even using the numbers within the list we were really just using it as almost a counter now let’s copy this integers once again let’s go right up here and let’s go copy this for Loop that we wrote now we do not have to call this number this can be anything you want any variable name that you’d like to name it we could call it jelly and we can do jelly plus jelly I think you’re getting the picture right when it Loops through that one it’s doing one plus one when it Loops through the two it’s doing 2 + 2 that is basically how a for Loop works now for a dictionary it’s going to handle it a little bit differently so let’s create a dictionary really quickly so we’ll say ice cream d iary is equal to we’re going to do a squiggly brackets so we’re going to say name and we’re going to say colon we need to assign our value for that item so we’re going to say Alex freeberg we’ll do our next one separated by a comma and we’ll say weekly intake and I’ll say five Scoops per week the next one we will do is favorite ice creams and for this one we’re going to do something a little bit different for this we’re going to have a list list within this dictionary so we’ll say within our list of my favorite ice creams we’ll say mint chocolate chip and I’ll just do MCC for that and we’ll separate that out by a comma and we’ll say chocolate so now we have this dictionary ice cream dick and within it we have my name my weekly intake and my favorite ice creams with a list in there as well let’s hit shift enter and now we’re going to start writing our for Loop now the for Loop is going to look very similar but to call it dictionary it’s just a little bit different so we’re going to say for the cream in icore creamore dictionary. values and then we’re going to do parentheses and then a colon now we’re going to print the cream so in order to indicate what we actually want to pull we have to specify within the dictionary what we want are we pulling the item are we pulling the value we need to specify this so that’s why we have thist value right here so let’s run this and see what we get so as you can see we are pulling in the values right here that’s why we’re pulling in Alex freeberg 5 and mint chocolate chip SL chocolate now we are able to call both of those both the key and the value so let’s go right down here and we can do both the key and the value so we can pull two things at one time and we’re going to do this by saying do items so we could also do key if we just wanted to do a key but we want to do items so we going to do both of them so we’re going to go right down here and say four key and value in ice cream dictionary. items print and let’s write key and then we’ll do a comma and then let’s give it a little arrow or something like that uh something like this and then we’ll do a comma and we’ll say value and let’s print this off and see what we get so it’s looping through and for each key and value it’s saying here is the key so that’s the name then we have weekly intake then we have favorite ice creams it’s giving us a little arrow and then we’re also printing off the value so we have name Alex freeberg weekly intake five favorite ice creams mint chocolate chip and chocolate so now let’s talk about nested for Loops we’ve looked at for Loops we understand how they work and why they do what they do but what about a nested for Loop a for Loop within a for Loop for this example let’s create two separate lists let’s create flavors and let’s make that a list by making it a bracket and we’ll do vanilla the classic chocolate and then cookie dough all great flavors so that’s our first list and then we’re going to say toppings and we’ll do a bracket for that as well and we’ll say fudge and then we’ll do Oreos and then we’ll do Marsh mows is how you spell marshmallows I think it’s an e that looks wrong I might be spelling it wrong but that’s okay so let’s save this by clicking shift enter and now we have our flavors and our toppings so now let’s write our first for Loops we’re going to say 41 as in our number one for loop we’re going to say in flavors and we’ll do a colon we’ll click enter now we can write our second for Loop so we’re going to say 42 in toppings and then we’ll do a colon and enter and then we’re going to say print and we’ll do an open parenthesis and then we’re going to say one so we’re printing the one in flavors and then we’re going to say one comma we to say topped with comma two so what this is essentially going to do is we’re going to say for one we’re going to take the very first one in flavors and then we’re going to Loop through all of two as well so we’re going to Loop through hot fudge Oreos and marshmallows and once we print that off then we will Loop all the way back to Flavors and look at the next iteration or the next sequence within the first for Loop so let’s run this really quickly and see what we get so as you can see it goes vanilla vanilla vanilla and vanilla is topped with the hot fudge the Oreos and the marshmallows and then we start iterating through our second one in our first for Loop so there’s that hierarchy so we’re iterating completely through this one before we actually go to the very first for Loop and start iterating through that one again now that is essentially how a nested for Loop works these nested for Loops can get very complicated in fact for Loops in general can get very complicated the more you add to it and the more you’re wanting to do with it but that is basically how a for Loop and a nested for Loop Works hello everybody today we’re going to be taking a look at while Loops in Python the while loop in Python is used to iterate over a block of code as long as the test condition is true now the difference between a for Loop and a while loop is that a for Loop is going to iterate over the entire sequence regardless of a condition but the while loop is only going to iterate over that sequence as long as a specific condition is met once that condition is not met the code is going to stop and it’s not going to iterate through the rest of the sequence so if we take a look at this flowchart right here we’re going to enter this while loop and we have a test condition right here the first time that this test condition comes back false it’s going to exit the while loop so let’s start actually writing out the code and see how this while loop works so let’s create a variable we’re just going to say number is equal to one and then we’ll say while and now we need to write our condition that needs to be met in order for our block of code beneath this to run so we’re going to say while number is less than five and then we’ll do colon enter and now this is our block of code we’re going to say print and then we’ll say number now what we need to do is basically create a counter we’re going to say number equals number + 1 if you’ve never done something like this it’s kind of like a counter most people start it at zero in fact let’s start it at zero and then each time it runs through this while loop it’s going to add one to this number up here and then it’s going to become a one a two a three each time it iterates through this while loop now once this number is no longer less than five it’ll break out of the while loop and it will no longer run so let’s run this really quick by hitting shift enter so it starts at zero and it’s going to say while the number is less than five print number so the first time that it runs through it is zero and so it prints zero and then it adds one to number and then it continues that y Loop right here and it keeps looping through this portion it never goes back up here to this line of code this is just our variable that we start with and then once this condition is no longer met once it is false then it’s going to break out of that code now that we basically know how a y Loop Works let’s look at something called a break statement so let’s copy this right down here and what we’re going to say is if number is equal to three we’re going to break now with the break statement we can basically Stop the Loop even if the while condition is true so while this number is less than five it’s going to continue to Loop through but now we have this break statement so it’s going to say if the number equals three we’re going to break out of this while loop but if this is false we’re going to continue adding to that number just like normal so let’s execute this so as you can see it only went to three instead of four like before because each time it was running through this y while loop it was checking if the number was equal to three and once it got to three this became true and then we broke out of this while loop the next thing that I want to look at and we’ll copy this right down here is an else statement much like an if statement but we can use the else statement with a while loop which runs the block of code and when that condition is no longer true then it activates the else statement so we’ll go right down here and we’ll say else and we’ll do a colon and enter and then we’ll say print and we’ll say no no longer less than five now because this if statement is still in there it will break so let’s say six and then we’ll run this and so it’s going to iterate through this block of code and once this statement is no longer true once we break out of it we’re going to go to our else statement now as long as this statement is true it’s going to continue to iterate through but once this condition is not met then it will go to our L statement and we’ll run that line of code now the L statement is only going to trigger if the Y Loop no longer is true if we have something like this if statement that causes it to break out of the while loop the lse statement will no longer work so let’s say if the number is three and we run this the L statement is no longer going to trigger so this body of code will not be run now the next thing that I want to look at is the continue statement if the continue statement is triggered it basically rejects all remaining statements in the current iteration of the loop and then we’ll go to the next iteration now to demonstrate this I’m going to change this break into a continue so before when we had the break if the number was equal to three it would stop all the code completely but when we change this to continue which we’ll do right now what it’s going to do is it’s no longer going to run through any of the subsequent code in this block of code it’s just going to go straight up to the beginning and restart our while loop so what’s going to happen when we run this is it’s going to come to three it’s going to become three it’s going to continue back into the while loop but it’s never going to have that number changeed to be added to one to continue with the while loop this will create an infinite Loop let’s try this really quickly and as you can see it’s going to stay three forever eventually this would time out but I’m just going to stop the code really quick so if we just change up the order of which we’re doing things we’re going to say there and we’re going to put this down here so what it’s going to do now instead of printing the number immediately and then adding the number later we’re going to add the number right away and then we’re going to say if it is three we’re going to continue and it’s going to print the number so let’s try executing this and see what happens so as you can see we no longer have the three in our output what it did was when we got to the number three it continued and didn’t execute this right here which prints off that number hello everybody today we’re going to be taking a look at functions in Python a function is a block of code which is only run when you call it so right here we’re defining our function and then this is our body of code that when we actually call it is going to be ran so right here we have our function call and all we’re doing is putting the function with the parenthesis es that is basically us calling that function and then we have our output throughout this video I’m going to show you how to write a function as well as pass arguments to that function and then a few other things like arbitrary arguments keyword arguments and arbitrary keyword arguments all these things are really important to know when you are using functions so let’s get started by writing our very first function together we’re going to start off by saying DF that is the keyword for defining a function then we can actually name our function and for this one we’re just going to do first underscore function and then we do an open parenthesis and then we’ll put a colon we’ll hit enter and it’ll automatically indent for us and this is where our body of code is going to go now within our body of code we can write just about anything and in this video I’m not going to get super Advanced we’re just going to walk through the basics to make sure that you understand how to use functions so for right now all we’re going to say is print we’ll do an open parenthesis we’ll do an apostrophe and we’ll say we did it and now we’re going to hit shift enter and this is not going to do anything at least you won’t see any output from this if we want to see the output or we actually want to run that function and some functions don’t have outputs but if we want to run that function what we have to do is just copy this and put it right down here and now we’re going to actually call our function so let’s go ahead and click shift enter and now we’ve successfully called our first function this function is about as simple as it could possibly be but now let’s take it up a notch and start looking at arguments so let’s go right down here and we’re going to say Define number underscore squared we’ll do a parenthesis and our colon as well now really quickly when you’re naming your function it’s kind of like naming a variable you can use something like X or Y but I tend to like to be a little bit more descriptive but now let’s take a look at passing an argument into a function the argument is going to be passed right here in the parenthesis so for us I’m just going to call it a number and then we’re going to hit enter and now we’ll write our body of code and all we’re going to do for this is type print and open parenthesis and we’ll say number and we’ll do two stars at least that’s what I call it a star and a two and what this is going to do is it’s going to take the number that we pass into our function it’s going to put it right here in our body of code and then for what we’re doing it’s going to put it to the power of two and so when the user or you run this and call this function this number is something that you can specify it’s an argument that you can input that will then be run in this body of code so let’s copy this right here and then put it right down here into this next cell and we’ll say five and so this five is going to be passed through into this function and be called right here for this print statement let’s run it and it should come out as I believe 25 that is my fault I forgot to actually run this block of code so I’m going to hit shift enter so now we’ve defined our function up here and now we can actually call it so now we’ll hit shift enter and we got our output of 25 now in this function we only called one argument but you can basically call as many arguments arents as you want you just have to separate them by commas so let’s copy this and we’ll put it right down here now we’ll say number squared uncore custom and then we’ll do number and then we’ll do power so now we can specify our number as well as the power that we want to raise it to so instead of having two which is what you call hardcoded we can now customize that and we’ll have power and now when we call this function we can specify the number and the power and both of those will go into this body of code and be run and we can customize those numbers so let’s copy this and we’ll say 5 to the power of three and let’s make sure I Ram this so let’s do shift enter and now we will call our function and let’s hit shift enter and we got 5 to the^ of 3 which is 125 and just one last thing to mention is if you have two arguments within your function and you are calling right here you have to pass in two arguments you can’t just have one so if we have a five right here it’s going to error out we have to specify both Arguments for it to work now let’s take a look at arbitrary arguments now arbitrary arguments are really interesting because if you don’t know how many arguments you want to pass through if you don’t know if it’s a one a two or a three you can specify that later when you’re calling the argument so you don’t have to do it upfront and know that information ahead of time so let’s define our function so we’re going to say Define and then we’re going to say number underscore args and we’ll do an open parenthesis and a colon now within our argument right here typically we would just specify here’s what our argument will be it will be number or it will be a word right but what we’re going to do is something called an arbitrary argument so it’s unknown so we’re going to put star and then we’ll say args now you will see something exactly like this typically if you’re looking at tutorials that’ll have star args in there or you’re looking at just a generic piece of code this is what it will look like but for us we’re going to actually put number so again we have the star and then we have our arbitrary argument right here and then we’ll hit enter and we’re going to say print open parentheses and this is where it’s going to get a little bit different so we’re going to say number and then we’re going to do an open bracket and let’s say zero and then we’ll do that times and then we’ll say number again with a bracket of one so in a little bit once we run this and then we call this number args function right here we’re going to need to specify the number zero and the number one that’s going to be called so let’s go ahead and run this and then we are going to call it and let’s say 5 comma 6 comma 1 2 8 so right up here we did not know how many arguments we were going to pass through it could be five it could be a thousand and we could also call in a tuple and that’s what this is right here we’re calling in a tup so what it’s going to do now is when it calls this number it’s going to call the very first within that tupal which will be that five and then it’ll also call in this number which will be the first position which is the six so let’s hit shift enter and it’s going to multiply these numbers together so five * 6 is equal to 30 now like I just said this is a tuple so we don’t actually have to write out these numbers like we just did we can pass through a tuple when we are actually calling this function let’s do that right up here let’s just create um let’s call it argor Tuple and we’ll do open parentheses and we’ll do the same numbers let’s just copy it make it easier and now we’ve created this tupal right here which we can then pass in and this is a lot more handy a lot more specific and this is most likely how someone would do something like this but let’s now create this and now we can copy AR Tuple and pass it through now really quickly this is going to fail and I’m doing that on purpose but I want to show you what you need to do in order to pass through this tupal so right now it’s going to say Tuple index is out of range all you have to do in order to use this is you have to specify a star before it just like you did when you creating your argument up here we have to put a star in front of our Tuple that we just passed through and now let’s try running this and now it works properly now the last two things that we’re going to look at are keyword arguments and arbitrary keyword arguments there are more things that you can learn and do within functions but again I’m just trying to teach you the basics to make sure that you understand how they work so let’s go right up here and a keyword argument is kind of similar to this right here and let’s actually copy this and put it right down here now a keyword argument is very similar in that you’re going to specify your arguments right here but what we did up here let me bring this down when we actually called the function what we did was we just put a five and a three and when we did that it automatically assigned number to five and power to three and that’s totally fine and you can do that but if you want a little bit more control you can use a keyword argument so right here we could say our is equal to five and number is equal to three so I just switched it around right number was assigned to five and power was assigned to three but I just switched it to show you how this might work let’s run both of these and now it’s 3 to the^ of 5 which is 243 so that essentially is a keyword argument again it just gives you a little bit more control you don’t have to put them in specific positions like if you’re just calling multiple arguments now let’s come right down here we’re going to create basically another custom function uh so for this one we’re going to write Define number underscore org and then we’ll do an open parenthesis a colon and enter and what this one is is is this one is a keyword argument or an arbitrary keyword argument now to specify an arbitrary argument all we did was a star and then we input number but if we’re doing a keyword argument we actually have to have two stars right here so let’s start taking a look and again if you’re doing arbitrary it means we don’t really know how many keyword arguments we want to pass into our function so we’re just going to put star star number and then later within our body of code and when we’re calling it we’ll be able to specify it and just like the arbitrary argument before the arbitrary keyword argument means we really just don’t know how many keyword arguments we’re going to need to pass into our function so to demonstrate this let’s write print do an open parenthesis and we’ll say my oops need to do an apostrophe my number is we’ll do just like that little space and we’ll say plus and this is kind of where it gets a little interesting or a little bit more tricky so what we’re going to say is number So This Is Us calling our number and then we’re going to do a bracket and then I’m actually going to go to calling the function it’s a little bit backward or a little bit different than what you might think but when we’re calling it what I’m going to do is I’m going to say integer is equal to let’s just do some random number now when we’re calling that keyword within our body of code what we’re going to do is we’re going to actually type out integer just like this and this looks a little bit different but what this this allows us to do is we can put as many keyword arguments in here as we want later and I’ll show you in just a second but for us we’re just creating this key and this value when we are calling it within the function so now when we create this and we run this oh whoops I forgot this has to be a string um so let’s run this again now we will say my number is 2309 then we’re going to add we’ll say plus and this isn’t going to look great but we’ll say my other number this will all be in the same line that’s okay my other number and then we’ll say number and we can specify again what we want in there so now we can go down here to where we’re calling it we’ll just put a comma and we’ll say integer oops integer 2 is equal to we’ll do a random number and then we’ll put integer two right here and then we’ll add plus right here so we don’t error out we’ll create this we’ll run this and as you can see both numbers were passed through again the syntax is terrible but now you can see that you have this arbitrary keyword argument right here and all we have to do is put number number and we can pass through as many of these arbitrary keyword arguments as we want as long as we just specify within our function when we’re calling it hello everybody today we’re going to be talking about converting data types in Python in this video I’m going to show you how to convert several different data types in including strings numbers sets tupal and even dictionaries so let’s start off by creating a variable we’ll say numor int is equal to 7 and we can check that data type by saying type and then inserting our variable number undor int and that will tell us that our data type for this variable is an integer let’s go ahead and create another one we’re going to say numor string is equal to and for this one we’ll also do a seven but let’s check the type and and we’ll do an open parenthesis and we’ll say the type of num string and that one is a string now let’s say we wanted to add those we’ll say num underscore sum so the sum of numor int plus numor string now when we’re adding these two values it is not going to work it’s going to give us an error and it’s going to say unsupported operand for INT and string so it cannot add both an integer and a string what we need to do in order to add these two numbers is to convert that string into an integer so let’s go right up here let’s add another cell and let’s say numor string undor converted is equal to and we want to convert it into an integer so all we have to do to convert it into an integer is type int and then we’re going to say numor string and that is as easy as it’s going to get all we have to do is say integer with our numb string inside of it and then it’s going to convert it and we can even check it right after by saying type num string converted and let’s run this and now we can see that it was converted into an integer so now let’s add that num string converted right here let’s copy and replace that string with the string converted and let’s actually print out that numor sum and it worked properly now we did not specify what type of value this Num Sum was going to be but because it was two integers in here it’s going to automatically apply that data type of integer to that num suum let’s go right down here and now let’s look at how we can convert lists sets and tupal so now let’s say we have a listor type and that’s equal to 1 2 3 and we can check it again by saying type and that is a list let’s say we want to convert it to a tuple it’s fairly easy all we’re going to do is write Tuple say listor type that listor type is now going to be a tupal and we can check that by saying type and wrapping it around this tupal and it shows us that it is converting that list into a tupal now we can also convert a list into a set but it may change the actual values within it let’s check that out really quickly so let’s say we have this list and let’s add a few more values to this just like that now let’s say we want to convert it to a set so we’re going to run this and we’ll say set of listor type and let’s try running this and see what the output is so this is something that you really need to be aware of when you are converting data types because set does not act the same as a list a set is basically going to take the unique values in the list and convert it to a set and it fundamentally changes the data that was in that original list and just to check the data type we can say type I’m just doing this for all of them and as you can see that is now a set now let’s go down here and take a look at dictionaries now let’s say we have a dictionary called dictionary type and we’ll do a squiggly bracket and we’ll say name and we’ll do a colon and we’ll say Alex then we’ll do age and a colon and we’ll say 28 and then we’ll do hair col and so really quickly let’s take that dictionary type and just confirm that it is a dictionary and it is and now what we’re going to do is take a look at all the items within that dictionary so we’re going to do dictionary type. items open parenthesis and this is going to show us all the items within it now we can also take this and look at something like the values and when we run that these are our values So within our dictionary we have items and that’s what this is right here this is one item and then within that we have our values which are right here so Alex 28 and Na and then we have something called a key and this is the key the name age and hair are all keys and we can look at that by saying dot keys so let’s say we want to take all of the keys and put that into a list what we’re going to do is we’re going to take this right here to say list we’ll do an open parenthesis we’ll type that in right there so it says a list and we’re converting these Keys into a list and let’s run that and now this is a list and let’s just check the type as well just to confirm and as you can see it was converted properly into a list and we can do the exact same thing with value Val and the values can also be converted into a list now we can also convert longer strings that aren’t just numbers like we did above in our very first example so let’s do longcore string and we’ll say I like to party now we’re going to take this string and we’re going to say list long string so we’re going to convert this string into a list and let’s see what happens so it took every single character in that string and put it into a list and we could also do a set as well that one’s a lot shorter because it’s only looking at unique values so that is how you convert data types in Python hello everybody today we’re going to be working on building a BMI calculator in Python now before we get started I want to show you this BMI calculator that I found online and it shows you the basic calculation that they use and that’s the one we’re going to use in this video and they also have this calculator right down here and some ranges that we can use for our calculator as well so for reference I weigh about 170 I’m about 59 let’s calculate this so I’m about a 25.1 BMI which falls into the overweight category that’s unfortunate but we can see exactly how this works and how RS should work when we actually build it so we’re going to kind of reference this throughout the video so let’s go right over here to our BMI calculator we need to calculate weight and height and then run this calculation right here so let’s go ahead and copy this and we’re going to put it right down here and so now we have our calculation so what we need is we need input from a user and there is an input function within python that we’re going to be using so let’s actually give me a few more cells so the first thing that we need to calculate is their weight let’s type out weight right here we’ll say weight is equal to and this is where we’ll use our input function so we’ll say input and when we actually run this it’s just going to give us this blank square or a user can input something we’ll say Alex so this is our output is what the actual user input and it does save it to this variable so if we say print weight it will still print out Alex now this is where we want the user to just like we did before where they’ll input their weight so we want to kind of give them a prompt for this we’ll put a string in here so I’ll do a double quote and then I’ll say enter your weight in and we’re using pounds say pounds colon space so now when we do this it’ll say enter your weight in pounds I’ll say 170 and then when we run this it does store that now let’s do print I should have saved it wait again oops now it’s only storing the value of 170 it’s not actually storing this string right here so that’s really important for when we do our calculations later um I’m going to I’m going to save this right down here because I’m sure I’m going to use that later um so we have that it’s working now we need to also do our height so let’s copy this and we’ll put it right here and we’ll do height and enter your height in inches so now for this one if we hit enter it’s actually running let’s stop it really quick and interrupt it let’s try running this so it’s going to say enter your weight and pounds that’s the first input say 170 and then when I hit enter it’s going to prompt me for that second input and so in inches 59 is 69 in and then I can hit enter again and now we have both of our inputs now we need this calculation right down here and just like that so now we have weight in pounds * 703 divided by height in inches by height in inches so we actually have weight and it’s already written in there but I’m just going to do it like this we’ll do weight time 703 so that’s pounds there our weight in pounds time 703 divided by now we have our height in inches times the height in inches so this is our calculation right here so let’s do this exact same thing let’s run this and this times of course is not going to work oops we need to do our star for both of these right now this is our calculation so let’s run this so we have 170 and that’s pounds and inches was 69 hit enter and it says cannot multiply the sequence of non- integer type of string Ah that’s because these are being stored in strings if right down here I do and we’ll do type of height we run that this is actually a string so we want to change that cuz we don’t need that anymore that so we don’t want it to be a string we need those to be integers or Floats or really anything besides a string it just needs to be numerical so integer float really so let’s do integer and we’ll wrap that input in it and we’ll do the same thing for this one now we have an integer for our weight an integer for our height so now when we’re running this calculation it should work properly let’s run this again our pounds are 70 our height is 69 in and it’s not giving us our output because we’re not printing anything okay so I just need to do print BMI so let’s try this again 1 70 69 and there is our BMI 25.1 so it worked the exact same as this one so they input well we input our height we inputed our or we inputed our weight we inputed our height and then it calculated rbmi the next thing that we need to do is we need to kind of give the user some context is that good is there BMI in within a good range a bad range we don’t know uh so let’s go ahead and I’m going to see if I can copy this know if this will work or not let’s go ahead and copy this right down here perfect so what we now need to do is we need to say okay if the user has given us this input we want to give them or tell them if they are a normal weight overweight obese severely obese anything like that and we have these ranges so that should help us out quite a bit so let’s just write our if statement and then we’ll include it up here but let’s go down here and we’ll say if and then we’ll do BMI and let’s just say BMI is greater than zero so if it’s greater than zero if they had any input where the BMI was not zero which should be every time if they do it properly don’t you know put a string in there or something or type out 40 which maybe we should make a prompt for that if that happens then we can say if we’ll do BMI and now we need to give that first range so this range right here so if it’s under 18.5 so we need to do a less than so if it’s less than 18.5 and it just says under it doesn’t say under or equal to so I’ll keep it at 18.5 so if it’s under 18.5 then let’s give kind of the output we’ll say print and the output or the basically the prompt is underweight so we’ll just say you are under under case underweight and just like that um then we’re going to pass several ellf statements through here well let’s just say else so I guess this would be like if they are if they don’t input something properly or something messes up maybe I we could write something like um print oops I’m thinking all this through we can write print enter valid inputs or something like this or we can always change that but let’s really quickly let’s run this okay so I’m not in that range uh let’s make the next one so then I can be within a certain range oops and we need we should need one more a minimum so we’ll say LF and LF these next two are this 24.9 so it’s going to check this one first so if it’s 18.5 or below 18.5 it’s automatically going to print this one so this next one we don’t have to do like a range or anything we can just say if it’s below if it’s between 25 and 20 9.9 so this one actually should be less than or equal to um this one is normal oh whoops 24.9 so this one is 24.9 this one is going to say you are normal weight so let’s run this now let’s see BMI was 25.1 oh guys I’m just messing up here I apologize all right this is the one that I was part of so now it’s going to be part of the overweight crowd now let’s run this and now our prompt is you are overweight because remember the BMI was saved right here as 25.1 down here if we run through this it’s saying no you’re not in oops get rid of that no you’re not in under 18.5 you’re not under 24.9 if you are under 29.9 you are overweight so that did work properly so that’s really good and I don’t think I want this to be our output for the person because we’re going to add this up here it’s just going to give us the BMI and then the output is going to say you are overweight uh let’s make it a little bit more customized um I’m going to say name is equal to input and then we’ll say enter your name um so it’ll be enter your name we’ll do Alex 70 69 there’s our BMI now it’s going to run through this logic or it will run through this logic in just a second when we actually finished this then we have 34.9 and let’s do one more oops and then this one’s going to be for 39.9 so this one was overweight this one is obese severely obese OB we’ll say severely you spell it severely obese and then anything that’s over that 40 and over so if it’s not this one anything else should be S morbidly obese so actually this else statement right here should say uh you are you are severely obese this is going to say morbidly morbidly obese now I added that name up here here because I wanted to add that down below actually so we’re going to say uh name plus and then we’ll do like comma you are underweight so it’ll be a little bit more personalized uh I think it’ll I think it’ll be a nice touch I really do we’ll do it like this and we’ll say you and let’s go back and do that to all of them and let me see how quickly I can do this oh whoops what I do got rid of that name plus u like that geez you guys are seeing me mess up a ton name plus you and then name plus you so now let’s run this and now it’s a little more personalized it says Alex you are overweight so this is all really good now this is an if statement um what we had done before I think is actually what we should put right down here so we’ll say else and then if that doesn’t work we’ll say what do we say enter valid input we’ll just put that um and let let me see if I can test this out don’t I don’t know if this will error out or if this will even work let me just see if I can mess with it and see if I can get it to work actually let’s copy this we’re going to copy this whole thing we’re going to include it right here and now we have basically our entire calculator so um let’s run this enter your name we’ll say Alex enter your pounds 170 Ander your inches 69 and then it’s going to say 25.1 Alex you are overweight and that’s perfect we could even go as far as adding like some feedback we could say you are overweight and then it would be a period and we could say um you need to exercise more stop sitting and writing so many python tutorials so now if we run this we’ll do Alex 17069 it says Alex you are overweight you need to exercise more and stop sitting and writing so many python tutorials period and that’s it this is the entire project um you can go a ton farther you can include much more complex logic you could even build out a UI to create your own you know app just like this where it has this input and this UI you can build that out with in jupyter notebooks with python um but that’s not really what this tutorial is for this is just to kind of help you um think through some of the logic of creating something like this hello everybody in this lesson we’re going to be taking a look at beautiful soup and requests now these packages in Python are really useful these are the two main ones that I use when I was first starting out with web scraping it can get a lot of what you want done in order to get that information out now of course there are other packages that you can use that may be a little bit more advanced but again this is just the beginner Series in a future series we’ll look at other packages as well that have some more advanced functionality so what we’re going to be doing is we’re going to import these packages and then we’re going to get all of the HTML from our website and make sure that it’s in a usable State and then in the next lesson we’re going to kind of query around in the HTML kind of pick and choose exactly what we want we’ll look at things like tags variable strings classes attributes and more so let’s get started by importing our packages what we’re going to say is from bs4 this is the module that we’re taking it from we’re going to say import and then we’ll do beautiful soup then we’re going to come down and we’re going to say import requests now let’s go ahead and run this I’m going to hit shift enter and it works well for me now if this do does not work for you you may potentially need to actually install bs4 so you may have to go to your terminal window and say pip install bs4 I’ll just let you Google how to do that if you need to do that CU it’s pretty easy but if you’re using Jupiter notebooks through Anaconda like how we set it up at the beginning of this python series then you should be totally fine it should be there for you the next thing that we need to do is specify where we’re taking this HTML from so what we need to actually do is come right over here to our web page and we need to get the URL so we’re going to go here we’re going to copy this URL and I’m just going to put it right here for a second and what we’re going to do is we’re going to be using this URL quite a bit so we just want to assign it to a variable so just say URL is equal to and then we’ll put it right in here now we can get rid of that so now this is our URL going forward this is where we’re going be pulling data from let’s go ahead and run this now we’re going to use requests and what we’re going to do is we’re going to say requests.get and then we’re going to put in url now this get function is going to use the request Library it’s going to send a get request to that URL and it’s going to return a response object let’s go ahead and run this as you can see here I got a response of 200 if you got something like a 204 or a 400 or 401 or 404 all of these things are potentially bad something like a 204 would mean there was no content in the actual web page 400 means a bad request so it was invalid the server couldn’t process it and you don’t get any response if you you got a 404 that might be one that you’re familiar with that’s an error that means the server cannot be found the next thing that we’re going to do is take the HTML now if you remember we come right back here and we inspect this we have all of this HTML right here now on this web page specifically right now it’s completely static it’s not a bunch of moving stuff or anything like that usually when you’re looking at HTML if you’re looking at something like Amazon and those web pages can update but when you actually pull that into python you’re basically getting a snapshot of the HTM at that time so what we’re going to do is bring in all of this HTML which is our snapshot of our website and then we can take a look at it so we’re going to come right down here and now we’re going to say beautiful soup so now we’ll use the beautiful soup package or Library so we need to say beautiful soup and we’re going do an open parenthesis we’re going to do two things there’s two parameters that we need to put in here first we need to put in this get request we actually need to name this and we’ll call this page we’ll say page is equal to and let’s run this and now we’re going to put that page in here and what we’re going to say is text so the page is what’s sending that request and then the text is what’s retrieving the actual raw HTML that we’re going to be using then we’re going to put a comma here and what we need to specify is how we’re going to parse this information now this is an HTML so what we’re going to do is HTML just like this this is a standard this already built into to this Library so we don’t need to go any further but it’s basically going to parse the information in an HTML format now let’s go ahead and run this let’s see what we get and as you can see we have a lot of information and as we scroll down I’ll try to point out some things that we’ve already looked at in previous lessons umm something like this th tag that should be very similar that’s the title then we have these TD tags and then of course if we scroll down even further we’ll have things like ATR tag so these are all things that we looked at in that first lesson when learning about HTML now again we want to assign this to a variable so we’re going to say soup that’s going to say equal to this information right here now I’m not going to go into all the history behind beautiful soup what I will say is the guy who created this beautiful soup Library uh what he said was is that it takes this really messy HTML or XML which you can also use it for and makes it into this kind of beautiful soup so I just thought that was kind of funny uh but that’s why we’re calling it soup right here and we’re going to go ahead and run this and we’ll come right down here here and we’ll say print soup and let’s run it and now we have everything in here so we have our HTML our head we have some HR and some links in here let scroll down a little bit more and then we have our body right there and of course we have a bunch of information in here now in the next lesson what we’re going to be doing is learning how to kind of query all of this to take specific information out and basically understand a lot of what’s going on in this HTML to make sure we can actually get what we need now if this looks really kind of messy to you and it just doesn’t make a lot of sense there is one more thing that I’m going to show you and we’ll come right down here so we’ll say soup. prettify and if you’ve ever used a different type of programming languages uh pry is very common in a lot of them where it’ll just make it a little bit more easy to visualize and see uh you’ll notice that it kind of has this hierarchy built in whereas if we scroll up there’s no hierarchy built in it’s all just down this left hand side so if you kind of want to view it and just kind of visually see the differences this does help a lot but it doesn’t actually help a lot when you’re you know querying it or using you know find and find all which is what we’re going to look at in the next lesson now the first thing that we need to learn is HTML HTML stands for hypertext markup language and it’s used to describe all of the elements on a web page now when we actually go to a website and start pulling data and information we need to know HTML so we can specify exactly what we want to take off of that website so that’s where HTML comes in and we’re going to look at the basics understanding just the basic structure of HTML then we’ll go look at a real website and you’ll kind of see that’s a little bit more difficult than what we just have right here but this is the basic building blocks to get to what the HTML actually looks like on a website now this is basically what HTML looks like we have these angled brackets with things like HTML head title body and then you’ll notice that at the end we’ll have a body and then we’ll have a body at the bottom this forward SL body denotes that this is the end of the body section in HTML so everything inside of this is within this body so there is this hierarchy within HTML we have HTML and HTML at the bottom which encapsulates all the HTML on the website then we have things like head and head body and body now Within These sections we usually have things like classes tags attributes text and all these other things things that we’ll get to in different lessons but one of the easiest ones to notice and look at are tags things like a P tag or a title tag now Within These tags because this is a super simple example we have these strings here my first web page and this is what’s called a variable string and this is actual text that we could take out of this web page now that you understand the super basics of HTML let’s actually go to our website and I’m going to have a link down below but it’s going to be this one right here this is basically just a website that you can you know practice web scraping on it’s called scrape the site.com and what we’re going to do is look at the HTML behind this web page and you can do this on any website that you go on so we’re going to right click we’re going to go down to inspect now right off the bat this looks a lot more complicated and a lot more complex than the very simple illustration that we’re looking at but let’s kind of roll this up just a little bit you’ll notice we have HTML and HTM at the bottom we have a head and there is the end of the head and then a body and the end of the body so in a super simple sense it is similar but just the information that’s within it is a lot more difficult now if we look at this title right here this is our title tag if we click this little arrow this is our drop- down you’ll notice that here we have the string hockey teams forms searching imagination now let’s say we didn’t know we didn’t want to click on that and go find it there’s something that’s super helpful within this inspection page that you can click on right here it says select an element in the page to inspect it so we’re going to click on that and as we go through our page and let’s click on this title it’s going to take us to exactly where this is in our HTML this is extremely helpful extremely useful for example let’s say the data I want is down here I want to take in the Boston Bruins I can click on it and it’s going to take me to where that is exactly in the HTML this is where we can start writing our web scraping script to specify okay I’m looking for a TR tag I’m looking for a TD tag I’m looking for the class called team this is all information and things that we can use to specify exactly what we want to pull out of our web page now there are other things that we didn’t really look at as well in just our simple illustration let’s come right over here there’s things like HRS now these are hyperlinks so if we went and then clicked on this this is just regular text but inside of it is this hyperlink where if we clicked on it it would take us to another website and typically that’s denoted by this hre right here then you’ll typically see things like a P tag which usually stands for a paragraph now the last thing that I want to show you while we’re here and we’re going to learn a lot more in the next several lessons but if we come right down here there is this actual entire table here and let’s try to find this table and I’m having trouble selecting the entire thing but let’s select this team name and if we look at this team name you can see that this is encapsulating the tables this table tag now these are super helpful because it takes in the entire table now if we wrap this up and we look just at this it says class table and then we have the end of this table tag now when we open it it’s going to have all of this information so as you can see as I’m highlighting over it we have these th tags and we have these TD tags and even these TR tags which is the individual data and this is something that we’ll look at when we’re actually scraping all of the data from this table in a few future lesson so this is how we can use HTML how we can inspect the web page and see exactly what’s going on kind of under the hood and then in future lessons we’ll see how we can use this HTML to specify exactly what data we want to pull out thank hello everybody in this lesson we’re going to be taking a look at find and find all really we’re going to be looking at a ton of different things in this lesson this is where we really start digging in seeing how we can extract specific information from our web page but in order to do that let’s set everything up where we actually bring in the HTML like we did in the last lesson and we’re just going to write all this out one more time just for practice if nothing else and then we’ll get into actually getting that information from the HTML so we’re going to start by saying from bs4 import beautiful soup there we go and import requests we’ll go ahead and run this then we’re going to come up here grab our HTML or sorry our URL we’ll say URL is equal to to and we’ll have that right here now we need to say page is equal to and then we’ll do requests.get and then we’ll put in our URL right here and we’re going to come over here and run this and lastly we need to say soup so we’ll say soup is equal to beautiful soup there we go and then within our parentheses we need to specify the page. text because we need that and our parser which is HTML and there we go and let’s go ahead and run this let’s print it out make sure it’s working and there we go so we have our soup right here all this should look really similar to uh our last lesson and so now we’ brought in our HTML from our page we have a lot a lot a lot of information in here now really quickly let’s come over and let’s inspect our web page now in here we have a ton of information right we have bunch of different tags and classes and all these other things but how do we actually use these well that’s where the find and find all is going to come into play and they’re pretty similar and you’ll see that in just a little bit but let’s say we want to take uh one of these tags and let’s come down let’s say we just want to take this div tag now there’s going to be a lot of different div tags in our HTML but let’s just come right here let’s go down and let’s say we’re going to call soup we’re going to say soup that’s all of our information we’re going to say do find now within our parentheses we can specify a lot of different things but we’re going to keep it really simple right now we’re just going to say di let’s go ahead and run this what this is going to bring up is the very first div tag in our HTML and that’s going to be this information right here now let’s copy this and we’re going to do the exact same thing except we’re going to say find underscore all now let’s run this now we’re going to have a ton more information really all find and find all do is that they find the information now find is only going to find the first response in our HTML lead that’s the div class container let’s go back up to the top that’s our div class container but find all is going to find all of them so it’ll put it in this list for you so it’s going to have this first one and it goes down to uh this word SL div which should be right here and then we have have a comma which separates our next div tag so that is how we can use it now what if we want to specify one of these div tags we pulled in a ton of them but we want to just look for one of them well this is something where the class comes in handy because right now we have classes equal to container class is equal to co md-12 I don’t know what these are at the off the top of my head but um usually they’ll be somewhat unique and we can use these to help us specify what we’re looking for for example just kind of glancing of this we could also use this a tag if we wanted to look at this so we could say oh we’re looking for uh these H refs so we have an hre here and this right down here we have this hre as well which again uh if you remember from previous lesson that stands for a hyperlink now something like the class or the href um or these IDs these are all attributes so we can specify or kind of filter Down based off of these now let’s try it so what we can do is we can do class first and this is kind of the default uh within something like find all is you can even do class underscore we can come right back up we have this div and then here’s our class so again we have to have the div and the class if we took this a tag this is an a tag which would go right here with the class of something like navlink or something like navlink again down here we need to specify that more but we have our div so we’ll say CL Cole md12 right here and let’s go ahead and run this and now it’s going to pull in just that information now we’re still getting a list because we have multiple of these so this div class uh Co md-12 doesn’t just happen once if we scroll down we’ll see it multiple times something like right here uh or actually let me see right here so here’s this comma then here’s our next one so we have two of these uh div tags with a class of coal- md-12 and in each of these we have different information this looks like a paragraph with this P tag right here and let’s scroll back up uh so I also think we should try out doing something like this P tag typically these P tags stand for paragraphs or they have text information in them let’s try to P tag really quickly let’s just see what we get and let’s run this and it looks like we get multiple P tags now if we come back here you can see that there’s this information and it’s this information that we’re pulling in and I’m just you know noticing that from right here and then we have this information right here and it looks like there’s one more which is this hre which looks like this open source so data via and then that uh hyperlink or that link right there so we have three different P tags now just to verify and make sure that that’s correct what we could do is come over here we’re going to click on this paragraph it’s going to take us to that P tag where the class is equal to lead let’s come over here and look at this paragraph now we have another P tag right over here where the class is equal to glyphicon glyphicon education I have no idea what that means um and then we’ll go to our last one which is right here where the P tag is equal to uh we have AAG hre class uh and a bunch of other information so let’s say we just wanted to pull in this paragraph right here let’s go here and see how we can specify this information so it looks like P or the class is equal to lead that looks like it’s going to be unique to just that one so if we come down here we’re going to say comma and it was class so you can do uh class underscore is equal to and then we’re going to say lead let’s try running this and we’re just pulling in that information now let’s say we actually want to pull in this paragraph We actually want this text right here and this is a very real use case you know let’s say I’m trying to pull in some information or or a paragraph of text well let’s copy this and what we’re going to then do is say. text and let’s run this now we’re going to get an error right here and this is a very common error because we’re trying to use find all unfortunately find all does not have a text attribute we actually need to change this to find typically when I’m working with these find and find alls I’m using find all most of the time until I want to start extracting text then when I specify it I’ll change this back to find just like this now let’s try this and now we’re getting in parentheses this information now this is all wonky it needs to definitely be cleaned up a little bit but if we Cod back up it’s no longer in a list and we no longer have things like these P tags in here or this class attribute so we’re really just trying to pull out this information now again this does not look perfect we could even trying to do something like strip look like there’s some white space uh that cleans it up a little bit this definitely looks a little better um and we could definitely go in here and clean this up more but just for you know an example this is how we can then extract that information now let’s look at one more example this is some information and this is what we’re going to do kind of our little mini project in the next lesson on let’s say we wanted to take all this information well what if we wanted to pull in something like the team name that’s going to be in right here in this TR tag and each of these TR tags have th tags underneath them so if we scroll down you’ll notice that each row is this TR tag so let’s go ahead and search for let’s do th let’s just search for that first so let’s come right back up here let’s use this find all and we’ll get rid of this text for right now and let’s just say we want to look for the TR is that what we said we were looking for no th so let’s say we’re looking for th let’s go ahead and run this so we’re going to have underneath this th we have team name year wins losses and notice these are all the titles so these titles are the only ones with these th tags if we go down you’ll notice that the data is actually TD tags so now let’s go back and look for TD we’ll say d and this is going to be a lot longer we have a lot of information but these are all the rows of data let’s see if we can just get one piece of this data we’re going to get back we want just this team name that’s all we’re trying to pull in for now um and then we’ll try to get this row and then in the next lesson we’re going to try to get all of this information make it look really nice and then we’ll put it into a panda’s data frame so let’s just get this team name right now let’s go ahead we’re going to say th let’s run this and we have this th and now that we know we’re getting this information in we can do find let’s run this so there’s our team name I’m just going to say. text and again we can do do strip just like that and Bam we have our team name so you can kind of start getting the idea of how we’re pulling this information out we’re really just specifying exactly what we’re seeing in this HTML and and what’s really really helpful and you know something that I do all the time is I’m inspecting it I’m just kind of searching like how what do I want what piece of information do I want then I go ahead and click on it and then I’m looking you know where is this sitting in the hierarchy it’s within the body it’s within this table with the class of table then it’s down here where this TR tag and then this TD tag so I’m looking kind of at the hierarchy and I’m specifying exactly what I’m looking for so that is what we’re going to look at in today’s lesson that’s how we can use f find and find all we were able to look at classes and tags and attributes and variable strings which is this right here getting that text uh and variable strings and we will look at find and find all and how it’s pulling that information in and how we can specify exactly what we’re looking for hello everybody in this lesson we are going to be scraping data from a real website and putting it into a panda’s data frame and maybe even exporting it to CSV if we’re feeling a bit spicy now in the last several lessons we’ve been looking at this page right here and I even promised that we were going to be pulling this data but as I was building out the project I just I honestly thought it was a little bit too easy since in the last lesson we kind of already pulled out some information from this table and I want to kind of throw you guys off so we’re going to be pulling from a different table we’re going to be going on to Wikipedia and looking at the list of the largest companies in the United States by revenue and we’re going to be pulling all of this information so if you thought this was going to be easy in a little mini project uh it’s now a full project because why not so let’s get started uh what we’re going to do is we’re going to import beautiful soup and requests we’re going to get this information and we’re going to see how we can do this and it’s going to get a little bit more complicated and a little bit more tricky we’re going to have to you know format things properly to get it into our Panda data frame to make it looking good and making it more usable so let’s go ahead and get rid of this easy table we don’t want that one uh and we’re going to come in here and we’re just going to start off this should look uh really familiar by now we’re going to say from bs4 import beautiful soup I don’t know if you’ve noticed but I’ve messed up spelling beautiful soup in every single uh video I’ve noticed uh let’s run this and now we need to go ahead and get our URL so let’s come up here let’s get our URL say URL is equal to and we’ll just keep it all in the same thing really quickly because we know this by Heart by now right uh we’ll say request. get and then URL to make sure that we’re getting that information it give us a response object um hopefully it’ll be 200 that’ll mean a good response and then we’ll say soup is equal to and then we’ll say beautiful soup and we’ll do our page. text now we’re pulling in the information from this URL and then we use our parser which will be oops HTML and let’s go ahead and run this looks like everything went well let’s print our soup now this is completely new to you it’s completely new to me I don’t know what I’m doing uh but it looks like we’re pulling in the information am I right so we got a lot of things going for us uh the uh stuff was imported properly we got our URL we got our soup which is uh not beautiful in my opinion but let’s keep on rolling let’s come right down here now what we need to do is we need to specify what data we’re looking for so let’s come and let’s inspect this web page now the only information that we’re going to want want is right in here we’re going to want these uh titles or these headers whoops so we’re going to want rank name industry Etc and then we are for sure going to want all of this information let’s just scroll down see if there’s anything tricky in here all right that looks pretty good uh and there is another table so there’s not just one table in here there are two tables in this page so that might change things for us but let’s come right back and let’s inspect our page by using this little button right here and let’s specify in let’s see if I can highlight just this page oh it’s not oh let’s do that right there so now we have this uh Wiki table sorter now I’m going to actually come right here I’m going to copy and I’m just going to say copy the outer HTML I’m just going to paste it in here real quick and that’s a ton of information I didn’t think it was going to copy all of it and we’re just going to delete that I just wanted to keep that class uh because I wanted to then come right down here at the bottom and just see what this table uh looks like I don’t know if it’s part of it or if it’s a if it’s its own table um I can’t tell let’s look at this Rank and let’s come up so it says uh it’s under this table and it looks like it’s its own table but it says Wiki table sort sortable jQuery table sorter what could be do a sortable jQuery table Ser so it looks like there are two tables with the same class which shouldn’t be a problem if we’re using find to get our text because we should be taking the first one which will be this table and this is the table we want um and if we wanted this one we could just use find all and since it’s a list we could use indexing to pull this table right um but I think we’re going to be okay with just pulling in this one so let’s go ahead and let’s do our find so we’ll do soup. find and we could find all or we could just do find a table let’s just try this and see what we get and if it pulls in the right one that we’re looking for that’ be great now this does not look correct at all um I don’t know what table it’s pulling in oh maybe it’s this right here this might be a table yeah it is so we have this uh box more citations so actually we are going to have to do exactly like what I was talking about uh let’s pull this and we well we could do comma class uh right here and let’s do both you know what this is a learning opportunity let’s do both so let me go back up to the top because I need these um and what we’re going to do let come right down here I want to add in uh another thing actually I’ll just push this one up there we go so we’re going to say findor all let’s run this so now we have multiple and again we got that weird one first but if we scroll down here’s our comma and then here’s our wik Wiki table sortable and then we have rank name industry all the ones that we were hoping to see and I guarantee you if you scroll all the way to the bottom um we’re going to see potentially Wells Fargo Goldman Sachs I’m pretty sure those are um let’s see yeah here we go like Ford motor Wells Fargo Goldman Soxs that’s this table right here so now we’re looking at the third table but again this is a list so we can use indexing on this and we’ll just choose not position zero because that’s this one right here which we did not like well now we’ll take position one let’s run this let’s go back up to the top and this is our table right here rank name industry this is the information that that we were actually wanting just to confirm rank name industry Etc so this is the information we’re wanting and we’re able to specify that with our find all and this is the information we want so we now want to make this the only information that we’re looking at so I’m just going to copy this we didn’t need to use our class for this one you could probably could have um but we could so let’s actually um put this right down here this will be our table we’ll say equal to but then I’ll come right here and I’m going to say soup. find this is just for demonstration purposes we do table comma glcore is equal to and then we’ll look at this right here whoops me do this let’s see if we get the correct output and let’s run this and looks like we’re getting a nun type object uh if I remember looks like the actual class is this right here so let’s run this instead and I got to get rid of the index there we go okay so we were able to pull it in just using the find so the find table class and it says Wiki table sortable at least that’s the HTML that we’re pulling in right here let me go back because I don’t I don’t know if that’s what I was seeing earlier let’s just get this rank let’s go back up where’s the rank go rank there we go so here’s our Rank and let’s go up to the table and there’s our class yeah and and that’s just uh to me that’s a little bit odd so it says Wiki table sortable jQuery Das table sorder right here but in our actual um in our actual python script that were running it was only pulling in the wiki table sortable so it wasn’t pulling in the jQuery dasht sorter why uh I’m not 100% sure but all things that we’re working through and we were able to uh we were able to figure out so we’re going to make this our table we’re going to say tables equal to uh soup. findall and let’s run this and if we print out our table we have this table now this is our only data that we are looking at now the first thing that I want to get is I want to get these titles or these headers right here that’s what we’re going to get first so let’s go in here we can just look in this information you can see that these are with these th tags and we can pull out those th tags really easily let’s come right down here we’re just going to say t and we can get rid of this let’s run this now these are our only th tags because everything else is a TR tag for these rows of data so these th tags are pretty unique which makes it really easy which is really great because then we can just do worldcore titles is equal to so now we have these titles but uh they’re not perfect but what we’re going to do is we’re going to Loop through it so I’m going to say worldcore titles and I’ll kind of walk through what I’m talking about is in a list and each one is Within These th tags so th and then there’s our um string that we’re trying to get so we can easily take this list and use list comprehension and we can do that right down here so I’m going to keep this to where we can see it um we’ll do worldcore tore titles that’s equal to now we’ll do our list comprehension should be super easy uh we’ll just say for title in worldcore titles and then what do we want we want title. text that’s it um because we’re just taking the text from each of these we’re just looping through and we’re getting rank then We’re looping through getting name looping through getting industry that’s it so let’s go and print our world table titles and see if it worked and it’s did uh this looks like it needs to be cleaned up just a little bit so let’s go ahead and do that while we’re here before we actually put it into the uh P’s data frame oops I just wanted uh I just wanted this actually so what we’re going to do is try to get rid of those back slash ends if we do dot strip that may actually not work yeah uh because this is a list what we need to do is we can actually do it dot. text. strip right here let’s try to do it in there there there we go so now we have uh this and now this world tables is good to go now I’m actually noticing one thing that may be odd yeah so we have rank name industry it goes to headquarters but then in here we’re getting rank name industry and then the profits which is from this table right here which we don’t want uh let’s scroll back up let’s kind of backtrack this and see where this happened we did find all table we’re looking at the first one right and then we’re doing [Music] headquarters uh so we’re doing print table ah okay I think I found the issue here and let’s backtrack again this is we’re working through this together we’re going to make mistakes uh the table is what we actually wanted to do we just did soup. findall th which is going to pull in that secondary table um jeez we were not thinking here um so now we need to do find all on the table not the soup because now we were looking at all of them oh what a rookie mistake okay uh let’s go back now let’s look at this now it’s just down to headquarters okay okay let’s go ahead and run this let’s run this now we just have headquarters now let’s run this now we are sitting pretty okay excuse my mistakes Hey listen you know if it happens to me it happens to you I promise you this is you know this is a project this a little U little project we’re creating here so we’re going to run into issues and that’s okay we’re figuring out as we go now what I want to do before we start pulling in all the data is I want to put this into our Panda data frame we’ll have the uh you know headers there for us to go so we won’t have to get that later and it just makes it easier uh in general trust me so we’re going to import pandas as PD let’s go ahead and run this and now we’re going to create our data frame so we’ll say PD dot now we have these world uh t titles so what we’re going to do is pd. data frame and then in here for our columns we’ll say that’s equal to the world table titles and let’s just go ahead and say that’s our data frame and call our data frame right here let’s run it there we go so we were able to pull out and extract those headers and those titles of these columns we’re able to put it into our data frame so we’re set up and we’re ready to go we’re rocking and rolling the next thing we need let’s go back up next thing we need is to start pulling in this data right here so we have to see how we can pull this data in now if you remember that we had those th tags those were our titles as you can see I’m highlighting over it but down here now we have these TD tags and those are all encapsulated within a TR tag so these TR represent the rows right then the D represents the data within those rows so R for rows D for data so let’s see how we can use that in order to get the information that we want so let’s go back up here just going to take this cuz again we’re only pulling from table not soup not soup what were we thinking um and let’s go ahead and let’s look at TR let’s run this now when we’re doing this TR these do come in with the headers so we’re going to have to later on we’re going to have to get rid of these we don’t want to pull those in um and have that as part of our data but if we scroll down there’s our Walmart um we have the location these are all with these TD tags and then of course it’s separated by a comma then we have our td2 so above we had our td1 so Row one row two Row three all the way down now we will easily be able to use this right because this is our column data and we can even call it that column underscore data is equal to we’ll run that um and what we’re going to do is we’re going to Loop through that cuz it was all in a list so we’re going to Loop through that information but instead of looking at the TR tag we’re going to look at the T D tag so let’s come right down here we’ll say for the row in column row and we’ll do a colon now we need to Loop through this we’ll do something like row. findor all and then what are we looking for we’re not looking for the TR looking for the TD and just for now let’s print this off see what this looks like apparently I didn’t run this uh column data that’s why and let’s run this and what we actually need to do is something almost exactly like this and I’m going to put it right below it um instead of printing this off because again this is all in a list we’re using find all so we’re we’re printing off another list which isn’t actually super helpful um for each of or all these data that we’re pulling in what we can do is we can call this uh the rowcor data and then we’ll put the row data in here so we’ll say four and we’ll say in row data so we’ll just say for the data in row data and we’ll take the data we’ll exchange that and now instead of uh World Table titles we can change this into uh individual row data right and now let’s print off the individual row data so it’s the exact same process that we were doing up here and that’s how we cleaned it up and got this and we may not need to strip but let’s just run this and see what we get there we go um and strip I’m sure was helpful let’s actually get rid of this yeah strip was helpful it’s the exact same thing that happened on the last one so let’s keep that actually let’s run this and now let’s just kind of glance glance at this information let’s look through it this looks exactly like the information that’s in the table let’s just confirm with this first one uh 25 uh two what am I saying 572 754 2.4 2300 57275 2.4 2200 so this looks exactly correct now we have to figure out a way to get this into our table because again these are all individual lists it’s not like we’re just you know putting all of this in at one time we can’t just take the entire table and plop it into um into the data frame we need a way to kind of put this in one at a time now if you’re just here for web scraping and you haven’t taken like my panda series that’s totally fine that’s not what we’re here for anyways um but what we can do we’ll have our individual row data and we’re going to put it in kind of one at a time now the reason we have to do that is because when we had it like this and let’s go back when we had it like this it’s printing out all of it but what it’s really doing and let’s get rid of it um what it’s really doing is it’s kind of doing it like this it’s printing it off one at a time and it’s only going to save that current row of data this last one it’s only going to save that as it’s looping through so what we actually want to do is every time it Loops through we append this information onto the data frame so as it goes through and eventually it’s going to end up with this one but as it goes through let’s run this as it goes through it puts this one in and then the next time it Loops through it puts this one in and the next time it Loops through Etc all the way down um so let’s see how we can do this so we have our data frame right here let’s get rid of this let’s bring our data frame in now again like I just mentioned if you don’t know pandas and you haven’t learned that uh you know go take my uh series on that it’s really good and we do something very similar to this in that Series so I’m not going to kind of walk through the entire logic um but there is something called l which stands for location when you’re looking at the index on a data frame and we’re going to use that to our advantage so we’re going to say the length of the data frame so we’re looking at how many rows are in this data frame and then we’re going to say that’s our length then we’re going to take that length and use it when we’re actually putting in this new information pretty um pretty cool so we’re going to say df.loc then a bracket and we’re putting in that length so we’re checking the length of our data frame each time it’s looping through and then we’re going to put the information in the next position that’s exactly what we’re doing let’s go ahead and put in the individual row data um so let’s just recap We’re looping through this TR this is our column data so these TR that’s our row of data then we’re as as We’re looping through it we’re doing find all and looking for TD tags that’s our individual data so that’s our row data then we’re taking that that data each piece of data and we’re getting out the text and we’re stripping it to kind of clean it and now it’s in a list for each individual row then we’re looking at our current data frame which has nothing in it right now we’re looking at the length of it and we’re appending each row of this information into the next position so let’s go ahead and run this it’s working it’s thinking and it looks like we got an issue and not set a row with mismatched columns now we’re encountering an issue not one that I got earlier but we’re going to cancel this out we’re going to figure this out together so let’s print off our individual row data let’s look at this this one is empty uh this is I’m almost certain is probably the issue um I didn’t encounter this issue when I wrote these uh when I wrote this lesson um but I’m almost certain that this is the issue right here so let’s do the column data but let’s start at position um let’s try one and not parentheses I need brackets because this is a list right so it should work and there we go so now that first one’s gone so now we just have the information I didn’t even think about that um just a second ago but I’m glad we’re running into it in case you ran into that uh issue let’s go ahead and try this again and it looked like it worked so let’s pull our data frame down I could have just wrote DF let’s pull our data frame down and now this is looking fantastic fantastic now um these three dots just mean there’s information in there just doesn’t want to display it but it looks like we have our rank we have our name have the industry revenue revenue growth employees and headquarters for every single one so this is perfect now this is exactly what I was hoping to get now you can go in and use pandas and manipulate this and change it and you know dive into all the information in there but we can also export this into a CSV if that’s what you’re wanting so we could easily do that by saying we’ll do DF do2 CSV and then within here we’re just going to do R and specify our file path so let’s come down here to our file path then we’ll go to our folder for our output so we’re just going to take this path and let me do it like that so I have this path in my one drive documents python web scraping folder for output so you know I already made this um and I’m just going to put this right down here now I do have to specify what we’re going to call this um we’ll just call this companies and then we have to say CSV that is very important now if we run this I already know just because uh we have this Rank and this index here we’re going to keep this index in the output not great uh but let’s run it let’s look at our output there’s our companies and when we pull this up as you can see this is not what we want because we have this extra thing right here now if we’re automating this this would get super annoying so what we’re going to do is go back and just say index equals false let’s go out of here and now we’re just going to come right down here we’re going to say comma index equals false and so it’s going to take this index and it’s not going to import or actually export it into the CSV now let’s go ahead and run this let’s pull up our folder one more time and let’s refresh just to make sure should be good and now this looks a lot better so we’re able to take all of that information and put it into a CSV and it’s all there so this is the whole project so if we scroll all the way back up let’s just kind of glance at what we did here scroll down we brought in our libraries and packages we specified our URL we brought in our soup um and then we tried to find our table now that took a little bit of uh testing out but we knew that the table was the second one so in position one so we took that table we were also able to specify it using find but then we use the class and of course we just wanted to work with that table that’s all the data we wanted so we specified this is our table and we worked with just our table going forward of course uh we encountered some small issues user errors on my end but we were able to get our world titles and we put those into our data frame right here using pandas then next we went back and we got all the row data and the individual data from those rows and we put it into our Panda’s data frame then we came below and we exported this into an actual CSV file so that is how we can use web scraping to get data from something like a table and put it into a panda’s data frame I hope that this lesson was helpful I know we encountered some issues that’s on my end and I apologize but if you run into those same issues hopefully that helped uh but I hope this was helpful and if you like this be sure to like And subscribe below I appreciate you I love you and I will see you in the next lesson [Music]

By Amjad Izhar
Contact: amjad.izhar@gmail.com
https://amjadizhar.blog

Affiliate Disclosure: This blog may contain affiliate links, which means I may earn a small commission if you click on the link and make a purchase. This comes at no additional cost to you. I only recommend products or services that I believe will add value to my readers. Your support helps keep this blog running and allows me to continue providing you with quality content. Thank you for your support!

Learn Python in Under 3 Hours Variables, For Loops, Web Scraping Full Project

Python Fundamentals: A Study Guide

Quiz

Quiz Answer Key

Essay Questions

Glossary of Key Terms

Python Programming and Web Scraping Tutorial

Python Programming: Basics and Fundamentals

Python Basics & Setup

Python Fundamentals & Usage

Creating Jupyter Notebooks with Anaconda

Python Code: Concepts and Web Scraping

Python Variable Assignment: A Comprehensive Guide

Python Data Types: Numeric, Boolean, Sequence, Set, and Dictionary

Web Scraping with Beautiful Soup and Requests

The Original Text

Discover more from Amjad Izhar Blog

Comments

Leave a comment Cancel reply

More posts

Jinnah of Pakistan by Stanley Wolpert – Study Notes

From Jinnah to Zia by Muhammad Munir Chief Justice of Pakistan – Study Notes

Kashmir Elections and BJP’s Setback – Study Notes

Israel-Palestine Conflict: A Critical Analysis – Study Notes

Learn Python in Under 3 Hours Variables, For Loops, Web Scraping Full Project

Python Fundamentals: A Study Guide

Quiz

Quiz Answer Key

Essay Questions

Glossary of Key Terms

Python Programming and Web Scraping Tutorial

Python Programming: Basics and Fundamentals

Python Basics & Setup

Python Fundamentals & Usage

Creating Jupyter Notebooks with Anaconda

Python Code: Concepts and Web Scraping

Python Variable Assignment: A Comprehensive Guide

Python Data Types: Numeric, Boolean, Sequence, Set, and Dictionary

Web Scraping with Beautiful Soup and Requests

The Original Text

Share this:

Discover more from Amjad Izhar Blog

Comments

Leave a comment Cancel reply

More posts

Jinnah of Pakistan by Stanley Wolpert – Study Notes

From Jinnah to Zia by Muhammad Munir Chief Justice of Pakistan – Study Notes

Kashmir Elections and BJP’s Setback – Study Notes

Israel-Palestine Conflict: A Critical Analysis – Study Notes