Author: Amjad Izhar

  • FastAPI Analytics API with Python and TimeScale

    FastAPI Analytics API with Python and TimeScale

    This series of tutorials guides viewers through building a comprehensive analytics API from scratch using Python and FastAPI. It focuses on creating a robust backend service capable of ingesting and storing time-series data efficiently using a PostgreSQL database optimized for this purpose, called TimescaleDB. The process involves setting up a development environment with Docker, defining data models and API endpoints, implementing data validation with Pydantic, and performing CRUD operations using SQLModel. Furthermore, the tutorials cover aggregating time-series data with TimescaleDB’s hyperfunctions and deploying the application to production using Timescale Cloud and Railway. Finally, it explores making the deployed API a private networking service accessible by other applications, like a Jupyter notebook server, for data analysis and visualization.

    Analytics API Microservice Study Guide

    Quiz

    1. What is the primary goal of the course as stated by Justin Mitchell? The main objective is to build an analytics API microservice using FastAPI. This service will be designed to receive and analyze web traffic data from other services.
    2. Why is TimeScaleDB being used in this project, and what is its relationship to PostgreSQL? TimeScaleDB is being used because it is a PostgreSQL extension optimized for time-series data and analytics. It enhances PostgreSQL’s capabilities for bucketing and aggregating data based on time.
    3. Explain the concept of a private analytics API service as described in the source. A private analytics API service means that the API will not be directly accessible from the public internet. It will only be reachable from internal resources that are specifically granted access.
    4. What is the significance of the code being open source in this course? The open-source nature of the code allows anyone to access, grab, run, and deploy their own analytics API whenever they choose. This gives users complete control and flexibility over their analytics infrastructure.
    5. Describe the purpose of setting up a virtual environment for Python projects. Creating a virtual environment isolates Python projects and their dependencies from each other. This prevents conflicts between different project requirements and ensures that the correct Python version and packages are used for each project.
    6. What is Docker Desktop, and what are its key benefits for this project? Docker Desktop is an application that allows users to build, share, and run containerized applications. Its benefits for this project include creating a consistent development environment that mirrors production, easy setup of databases like TimeScaleDB, and simplified sharing and deployment of the API.
    7. Explain the function of a Dockerfile in the context of containerizing the FastAPI application. A Dockerfile contains a set of instructions that Docker uses to build a container image. These instructions typically include specifying the base operating system, installing necessary software (like Python and its dependencies), copying the application code, and defining how the application should be run.
    8. What is Docker Compose, and how does it simplify the management of multiple Docker containers? Docker Compose is a tool for defining and managing multi-container Docker applications. It uses a YAML file to configure all the application’s services (e.g., the FastAPI app and the database) and allows users to start, stop, and manage them together with single commands.
    9. Describe the purpose of API routing in a RESTful API like the one being built. API routing defines the specific URL paths (endpoints) that the API exposes. It dictates how the API responds to different requests based on the URL and the HTTP method (e.g., GET, POST) used to access those endpoints.
    10. What is data validation, and how is Pydantic (and later SQLModel) used for this purpose in the FastAPI application? Data validation is the process of ensuring that incoming and outgoing data conforms to expected formats and types. Pydantic (and subsequently SQLModel, which is built on Pydantic) is used to define data schemas, allowing FastAPI to automatically validate data against these schemas and raise errors for invalid data.

    Essay Format Questions

    1. Discuss the benefits of using a microservice architecture for an analytics API, referencing the technologies chosen in this course (FastAPI, TimeScaleDB, Docker).
    2. Explain the process of setting up a development environment that closely mirrors a production environment, as outlined in the initial sections of the course, and discuss the rationale behind each step.
    3. Compare and contrast the roles of virtual environments and Docker containers in isolating and managing application dependencies and runtime environments.
    4. Describe the transition from using Pydantic schemas for data validation to SQLModel for both validation and database interaction, highlighting the advantages of SQLModel in building a data-driven API.
    5. Explain the concept of Time Series data and how TimeScaleDB’s hypertable feature is designed to efficiently store and query this type of data, including the significance of chunking and retention policies.

    Glossary of Key Terms

    • API (Application Programming Interface): A set of rules and protocols that allows different software applications to communicate and exchange data with each other.
    • Microservice: An architectural style that structures an application as a collection of small, independent services, each focusing on a specific business capability.
    • Analytics API: An API specifically designed to provide access to and facilitate the analysis of data.
    • Open Source: Software with source code that is freely available and can be used, modified, and distributed by anyone.
    • FastAPI: A modern, high-performance web framework for building APIs with Python, based on standard Python type hints.
    • TimeScaleDB: An open-source time-series database built as a PostgreSQL extension, optimized for high-volume data ingestion and complex queries over time-series data.
    • PostgreSQL: A powerful, open-source relational database management system known for its reliability and extensibility.
    • Time Series Data: A sequence of data points indexed in time order.
    • Jupyter Notebook: An open-source web-based interactive development environment that allows users to create and share documents containing live code, equations, visualizations, and narrative text.
    • Virtual Environment: An isolated Python environment that allows dependencies for a specific project to be installed without affecting other Python projects or the system-wide Python installation.
    • Docker: A platform for building, shipping, and running applications in isolated environments called containers.
    • Container: A lightweight, standalone, and executable package of software that includes everything needed to run an application: code, runtime, system tools, libraries, and settings.
    • Docker Desktop: A user-friendly application for macOS and Windows that enables developers to build and share containerized applications and microservices.
    • Dockerfile: A text document that contains all the commands a user could call on the command line to assemble a Docker image.
    • Docker Compose: A tool for defining and running multi-container Docker applications. It uses a YAML file to configure the application’s services.
    • API Endpoint: A specific URL that an API exposes and through which client applications can access its functionalities.
    • REST API (Representational State Transfer API): An architectural style for designing networked applications, relying on stateless communication and standard HTTP methods.
    • API Routing: The process of defining how the API responds to client requests to specific endpoints.
    • Data Validation: The process of ensuring that data meets certain criteria, such as type, format, and constraints, before it is processed or stored.
    • Pydantic: A Python data validation and settings management library that uses Python type hints.
    • SQLModel: A Python library for interacting with SQL databases, built on Pydantic and SQLAlchemy, providing data validation, serialization, and database integration.
    • SQLAlchemy: A popular Python SQL toolkit and Object Relational Mapper (ORM) that provides a flexible and powerful way to interact with databases.
    • Database Engine: A software component that manages databases, allowing users to create, read, update, and delete data. In SQLModel, it refers to the underlying SQLAlchemy engine used to connect to the database.
    • SQL Table: A structured collection of data held in a database, consisting of columns (attributes) and rows (records).
    • Environment Variables: Dynamic named values that can affect the way running processes will behave on a computer. They are often used to configure application settings, such as database URLs.
    • Hypertable: A core concept in TimeScaleDB, representing a virtual continuous table across all time and space, which is internally implemented as multiple “chunk” tables.
    • Chunking: The process by which TimeScaleDB automatically partitions hypertable data into smaller, more manageable tables based on time and optionally other partitioning keys.
    • Retention Policy: A set of rules that define how long data should be kept before it is automatically deleted or archived.
    • Time Bucket: A function in TimeScaleDB that aggregates data into specified time intervals.
    • Gap Filling: A technique used in time-series analysis to fill in missing data points in a time series, often by interpolation or using a default value.
    • Railway: A cloud platform that simplifies the deployment and management of web applications and databases.

    Briefing Document: Analytics API Microservice Development

    This briefing document summarizes the main themes and important ideas from the provided source material, which outlines the development of an open-source analytics API microservice using FastAPI, PostgreSQL with the TimescaleDB extension, and various other modern technologies.

    Main Themes:

    • Building a Private Analytics API: The primary goal is to create a microservice that can ingest and analyze web traffic data from other services within a private network.
    • Open Source and Customization: The entire codebase is open source, allowing users to freely access, modify, and deploy their own analytics API.
    • Modern Technology Stack: The project utilizes FastAPI (a modern, fast web framework for building APIs), PostgreSQL (a robust relational database), TimescaleDB (a PostgreSQL extension optimized for time-series data and analytics), Docker (for containerization), and other related Python packages.
    • Time-Series Data Analysis: A key focus is on leveraging TimescaleDB’s capabilities for efficient storage and aggregation of time-stamped data.
    • Production-Ready Deployment: The course aims to guide users through the process of deploying the analytics API into a production-like environment using Docker.
    • Local Development Environment Setup: A significant portion of the initial content focuses on setting up a consistent and reproducible local development environment using Python virtual environments and Docker.
    • API Design and Routing: The process includes designing API endpoints, implementing routing using FastAPI, and handling different HTTP methods (GET, POST).
    • Data Validation with Pydantic and SQLModel: The project utilizes Pydantic for data validation of incoming and outgoing requests and transitions to SQLModel for defining database models, which builds upon Pydantic and SQLAlchemy.
    • Database Integration with PostgreSQL and TimescaleDB: The briefing covers setting up a PostgreSQL database (initially standard, then upgraded to TimescaleDB) using Docker Compose and integrating it with the FastAPI application using SQLModel.
    • Querying and Aggregation of Time-Series Data: The document introduces the concept of time buckets in TimescaleDB for aggregating data over specific time intervals.
    • Cloud Deployment with Railway: The final section demonstrates deploying the analytics API and a Jupyter Notebook server (for accessing the API) to the Railway cloud platform.

    Most Important Ideas and Facts:

    • Open Source Nature: “thing about all of this is all the code is open source so you can always just grab it and run with it and deploy your own analytics API whenever you want to that’s kind of the point”. This emphasizes the freedom and customizability offered by the project.
    • FastAPI for API Development: The course uses FastAPI for its speed, ease of use, and integration with modern Python features. “The point of this course is to create an analytics API microservice we’re going to be using fast API so that we can take in a bunch of web traffic data on our other services and then we’ll be able to analyze them as we see fit.”
    • TimescaleDB for Time-Series Optimization: TimescaleDB is crucial for efficiently handling and analyzing time-based data. “we’ll be able to analyze them as we see fit and we’ll be able to do this with a lot of modern technology postgres and also specifically time scale so we can bucket data together and do aggregations together.” It enhances PostgreSQL specifically for time-series workloads. “Time scale is a postgres database so it’s still postres SQL but it’s optimized for time series.”
    • Private API Deployment: The deployed API is intended to be private and only accessible to authorized internal resources. “This is completely private as in nobody can access it from the outside world it’s just going to be accessible from the resources we deem should be accessible to it…”
    • Data Aggregation: The API will allow for aggregating raw web event data based on time intervals. “from this raw data we will be able to aggregate this data and put it into a bulk form that we can then analyze.”
    • Time-Based Queries: TimescaleDB facilitates querying data based on time series. “…it’s much more about how to actually get to the point where we can aggregate the data based off of time based off of Time series that is exactly what time scale does really well and enhances Post cres in that way.”
    • SQLModel for Database Interaction: The project utilizes SQLModel, which simplifies database interactions by combining Pydantic’s data validation with SQLAlchemy’s database ORM capabilities. “within that package there is something called SQL model which is a dependency of the other package…this one works well with fast API and you’ll notice that it’s powered by pantic and SQL Alchemy…”
    • Importance of Virtual Environments: Virtual environments are used to isolate Python project dependencies and avoid version conflicts. “we’re going to create a virtual environment for python projects so that they can be isolated from one another in other words python versions don’t conflict with each other on a per project basis.”
    • Docker for Environment Consistency: Docker is used to create consistent and reproducible environments for both development and production. “docker containers are that something what we end up doing is we package up this application into something called a container image…”
    • Docker Compose for Multi-Container Management: Docker Compose is used to define and manage multi-container Docker applications, such as the FastAPI app and the PostgreSQL/TimescaleDB database.
    • API Endpoints and Routing: FastAPI’s routing capabilities are used to define API endpoints (e.g., /healthz, /api/events) and associate them with specific functions and HTTP methods (GET, POST).
    • Data Validation with Pydantic: Pydantic models (Schemas) are used to define the structure and data types of request and response bodies, ensuring data validity.
    • Transition to SQLModel for Database Models: Pydantic schemas are transitioned to SQLModel models to map them to database tables. “we’re going to convert our pantic schemas into SQL models…”
    • Database Engine and Session Management: SQLAlchemy’s engine is used to connect to the database, and a session management function is implemented to handle database interactions within FastAPI routes.
    • Creating Database Tables with SQLModel: SQLModel’s create_all function is used to automatically create database tables based on the defined SQLModel models. “This is going to be our command to actually make sure that our database tables are created and they are created based off of our SQL model class to create everything so it’s SQL model. metadata. create all and it’s going to take in the engine as argument…”
    • TimescaleDB Hypertable Creation: To optimize for time-series data, standard PostgreSQL tables are converted into TimescaleDB hypertables. “it’s time to create hyper tables now if we did nothing else at this point and still use this time scale model it would not actually be optimized for time series data we need to do one more step to make that happen a hypertable is a postgress table that’s optimized for that time series data with the chunking and also the automatic retention policy where it’ll delete things later.”
    • Hypertable Configuration (Chunking and Retention): TimescaleDB hypertables can be configured with chunk time intervals and data retention policies (drop after). “the main three that I want to look at are these three first is the time column this is defaulting to the time field itself right so we probably don’t ever want to change the time colum itself although it is there it is supported if you need to for some reason in time scale in general what you’ll often see the time column that’s being used is going to be named time or the datetime object itself so that one’s I’m not going to change the ones that we’re going to look at are the chunk time interval and then drop after…”
    • Time Bucketing for Data Aggregation: TimescaleDB’s time_bucket function allows for aggregating data into time intervals. “we’re now going to take a look at time buckets time buckets allow us to aggregate data over a Time interval…”
    • Cloud Deployment on Railway: The project demonstrates deploying the API and a Jupyter Notebook to the Railway cloud platform for accessibility.

    Quotes:

    • “thing about all of this is all the code is open source so you can always just grab it and run with it and deploy your own analytics API whenever you want to that’s kind of the point”
    • “The point of this course is to create an analytics API microservice we’re going to be using fast API so that we can take in a bunch of web traffic data on our other services and then we’ll be able to analyze them as we see fit and we’ll be able to do this with a lot of modern technology postgres and also specifically time scale so we can bucket data together and do aggregations together”
    • “This is completely private as in nobody can access it from the outside world it’s just going to be accessible from the resources we deem should be accessible to it”
    • “from this raw data we will be able to aggregate this data and put it into a bulk form that we can then analyze”
    • “Time scale is a postgres database so it’s still postres SQL but it’s optimized for time series”
    • “within that package there is something called SQL model which is a dependency of the other package…this one works well with fast API and you’ll notice that it’s powered by pantic and SQL Alchemy…”
    • “we’re going to create a virtual environment for python projects so that they can be isolated from one another in other words python versions don’t conflict with each other on a per project basis”
    • “docker containers are that something what we end up doing is we package up this application into something called a container image…”
    • “it’s time to create hyper tables now if we did nothing else at this point and still use this time scale model it would not actually be optimized for time series data we need to do one more step to make that happen a hypertable is a postgress table that’s optimized for that time series data with the chunking and also the automatic retention policy where it’ll delete things later.”
    • “we’re now going to take a look at time buckets time buckets allow us to aggregate data over a Time interval…”

    This briefing provides a comprehensive overview of the source material, highlighting the key objectives, technologies, and processes involved in building the analytics API microservice. It emphasizes the open-source nature, the use of modern tools, and the focus on efficient time-series data analysis with TimescaleDB.

    Building a Private Analytics API with FastAPI and TimescaleDB

    Questions

    • What is the main goal of this course and what technologies will be used to achieve it? The main goal of this course is to guide users through the creation of a private analytics API microservice. This API will be designed to ingest and analyze web traffic data from other services. The course will primarily use FastAPI as the web framework, PostgreSQL as the database, and specifically TimescaleDB as a PostgreSQL extension optimized for time-series data. Other tools mentioned include Docker for containerization, Cursor as a code editor, and a custom Python package called timescaledb-python.
    • Why is the created analytics API intended to be private, and how will it be accessed and tested? The analytics API is designed to be completely private, meaning it will not be directly accessible from the public internet. It will only be reachable from internal resources that are specifically granted access. To test the API after deployment, a Jupyter Notebook server will be used. This server will reside within the private network and will send simulated web event data to the API for analysis and to verify its functionality.
    • What is TimescaleDB, and why is it being used in this project instead of standard PostgreSQL? TimescaleDB is a PostgreSQL extension that optimizes the database for handling time-series data. While it is built on top of PostgreSQL and retains all of its features, TimescaleDB enhances its capabilities for data bucketing, aggregations based on time, and efficient storage and querying of large volumes of timestamped data. It is being used in this project because the core of an analytics API is dealing with data that changes over time, making TimescaleDB a more suitable and performant choice for this specific use case compared to standard PostgreSQL.
    • Why is the course emphasizing open-source tools and providing the code on GitHub? A key aspect of this course is the commitment to open-source technology. All the code developed throughout the course will be open source, allowing users to freely access, use, modify, and distribute it. The code will be hosted on GitHub, providing a collaborative platform for users to follow along, contribute, and deploy their own analytics API based on the course materials. This also ensures transparency and allows users to have full control over their analytics solution.
    • What are virtual environments, and why is setting one up considered an important first step in the course? Virtual environments are isolated Python environments that allow users to manage dependencies for specific projects without interfering with the global Python installation or other projects. Setting up a virtual environment is crucial because it ensures that the project uses the exact Python version and required packages (like FastAPI, Uvicorn, TimescaleDB Python) without conflicts. This leads to more reproducible and stable development and deployment processes, preventing issues caused by differing package versions across projects.
    • What is Docker, and how will it be used in this course for development and potential production deployment? Docker is a platform that enables the creation and management of containerized applications. Containers package an application along with all its dependencies, ensuring consistency across different environments. In this course, Docker will be used to set up a local development environment that closely mirrors a production environment. This includes containerizing the FastAPI application and the TimescaleDB database. Docker Compose will be used to manage these multi-container setups locally. While the course starts with the open-source version of TimescaleDB in Docker, it suggests that for production, more robust services directly from Timescale might be considered.
    • How will FastAPI be used to define API endpoints and handle data? FastAPI will serve as the web framework for building the analytics API. It will be used to define API endpoints (URL paths) that clients can interact with to send data and retrieve analytics. FastAPI’s features include automatic data validation, serialization, and API documentation based on Python type hints. The course will demonstrate how to define routes for different HTTP methods (like GET and POST) and how to use Pydantic (and later SQLModel) for defining data models for both incoming requests and outgoing responses, ensuring data consistency and validity.
    • How will SQLModel be integrated into the project, and what benefits does it offer for interacting with the database? SQLModel is a Python library that combines the functionalities of Pydantic (for data validation and serialization) and SQLAlchemy (for database interaction). It allows developers to define database models using Python classes with type hints, which are then automatically translated into SQL database schemas. In this course, SQLModel will be used to define the structure of the event data that the API will collect and store in the TimescaleDB database. It will simplify database interactions by providing an object-relational mapping (ORM) layer, allowing developers to work with Python objects instead of writing raw SQL queries for common database operations like creating, reading, updating, and deleting data.

    Analytics API for Time Series Data with FastAPI and TimescaleDB

    Based on the source “01.pdf”, an Analytics API service is being built to ingest, store, and refine data, specifically time series data, allowing users to control and analyze it. The goal is to create a microservice that can take in a lot of data, such as web traffic data from other services, store it in a database optimized for time series, and then enable analysis of this data.

    Here are some key aspects of the Analytics API service as described in the source:

    • Technology Stack:
    • FastAPI: This is used as the microservice framework and the API endpoint, built with Python. FastAPI is described as a straightforward and popular tool that allows for writing simple Python functions to handle API endpoints. It’s noted for being minimal and flexible.
    • Python: The service is built using Python, which is a dependency for FastAPI. The tutorial mentions using Python 3.
    • TimescaleDB: This is a PostgreSQL database optimized for time series data and is used to store the ingested data. TimescaleDB enhances PostgreSQL for time series operations, making it suitable for bucketing data and performing aggregations based on time. The course partnered with Timescale.
    • Docker: Docker is used to containerize the application, making it portable and able to be deployed anywhere with a Docker runtime, and also to easily run database services like TimescaleDB locally. Containerization ensures the application is production-ready.
    • Railway: This is a containerized cloud platform used for deployment, allowing for a Jupyter Notebook server to connect to the private analytics API endpoint.
    • Functionality:
    • Data Ingestion: The API is designed to ingest a lot of data, particularly web traffic data, which can change over time (time series data).
    • Data Storage: The ingested data is stored in a Timescale database, which is optimized for efficient storage and querying of time-based data.
    • Data Modeling: Data modeling is a key aspect, especially when dealing with querying the data. The models are based on Pydantic, which is also the foundation for SQLModel, making them relatively easy to work with, especially for those familiar with Django models. A custom Python package, timescale-db-python, was created for this series to facilitate the use of FastAPI and SQLModel with TimescaleDB.
    • Querying and Aggregation: The service allows for querying and aggregating data based on time and other parameters. TimescaleDB is particularly useful for time-based aggregations. More advanced aggregations can be performed to gain complex insights from the data.
    • API Endpoints: FastAPI is used to create API endpoints that handle data ingestion and querying. The tutorial covers creating endpoints for health checks, reading events (with list and detail views), creating events (using POST), and updating events (using PUT).
    • Data Validation: Pydantic is used for both incoming and outgoing data validation, ensuring the data conforms to defined schemas. This helps in hardening the API and ensuring data integrity.
    • Development and Deployment:
    • Local Development: The tutorial emphasizes setting up a local development environment that closely mirrors a production environment, using Python virtual environments and Docker.
    • Production Deployment: The containerized application is deployed to Railway, a containerized cloud service. The process involves containerizing the FastAPI application with Docker and configuring it for deployment on Railway.
    • Private API: The deployment on Railway includes the option to create a private analytics API service, accessible only from within designated resources, enhancing security.
    • Example Use Case: The primary use case demonstrated is building an analytics API to track and analyze web traffic data over time. This involves ingesting data about page visits, user agents, and durations, and then aggregating this data based on time intervals and other dimensions.

    The tutorial progresses from setting up the environment to building the API with data models, handling different HTTP methods, implementing data validation, integrating with a PostgreSQL database using SQLModel, optimizing for time series data with TimescaleDB (including converting tables to hypertables and using time bucket functions for aggregation), and finally deploying the application to a cloud platform. The source code for the project is open source and available on GitHub.

    FastAPI for Analytics Microservice API Development

    Based on the source “01.pdf” and our previous discussion, FastAPI is used as the microservice framework for building an analytics API service from scratch. This involves creating an API endpoint that primarily ingests a single data model.

    Here’s a breakdown of FastAPI’s role in the context of this microservice:

    • API Endpoint Development: FastAPI is the core technology for creating the API endpoints. The source mentions that the service will have “well mostly just one data model that we’re going to be ingesting” through these endpoints. It emphasizes that FastAPI allows for writing “fairly straightforward python functions that will handle all of the API inp points”.
    • Simplicity and Ease of Use: FastAPI is described as a “really really popular tool to build all of this out” and a “really great framework to incrementally add things that you need when you need them”. It’s considered “very straightforward” and “minimal”, meaning it doesn’t include a lot of built-in features, offering flexibility. The models built with FastAPI (using SQLModel, which is based on Pydantic) are described as “a little bit easier to work with than Django models” with “less to remember” because they are based on Pydantic.
    • URL Routing: Similar to other web frameworks, FastAPI facilitates easy URL routing, which is essential for defining the different endpoints of the API service.
    • Data Validation and Serialization: FastAPI leverages Pydantic for data modeling. This allows for defining data structures with type hints, which FastAPI uses for automatic data validation and serialization (converting Python objects to JSON and vice versa). We saw examples of this with the Event schema and how incoming and outgoing data were validated.
    • HTTP Method Handling: FastAPI makes it straightforward to define functions that handle different HTTP methods (e.g., GET for retrieving data, POST for creating data, PUT for updating data) for specific API endpoints.
    • Middleware Integration: FastAPI allows the integration of middleware, such as CORS (Cross-Origin Resource Sharing) middleware, which was added to control which websites can access the API.
    • Integration with Other Tools: FastAPI works well with other tools in the described architecture, including:
    • SQLModel: Built on top of Pydantic and SQLAlchemy, SQLModel simplifies database interactions and is used within the FastAPI application to interact with the PostgreSQL database (TimescaleDB).
    • Uvicorn and Gunicorn: Uvicorn is used as an ASGI server to run the FastAPI application, particularly during development. Gunicorn can be used in conjunction with Uvicorn for production deployments.
    • Docker: FastAPI applications can be easily containerized using Docker, making them portable and scalable.
    • Production Readiness: The tutorial emphasizes building applications that are production-ready from the start, and FastAPI, along with Docker and Railway, facilitates this. The deployment process to Railway involves containerizing the FastAPI application.

    In essence, FastAPI serves as the foundation for building the API interface of the analytics microservice, handling requests, processing data according to defined models, and interacting with the underlying data storage (TimescaleDB) through SQLModel. Its design principles of being high-performance, easy to use, and robust make it well-suited for building modern microservices.

    TimescaleDB: PostgreSQL for Time Series Data

    Based on the source “01.pdf” and our conversation history, TimescaleDB is a PostgreSQL database extension that is optimized for time series data. While it is still fundamentally PostgreSQL, it includes specific features and optimizations that make it particularly well-suited for handling and analyzing data that changes over time, such as the web traffic data for the analytics API service being built.

    Here’s a more detailed discussion of PostgreSQL with the Timescale extension:

    • Based on PostgreSQL: TimescaleDB is not a separate database but rather an extension that runs within PostgreSQL. This means that it retains all the reliability, features, and the extensive ecosystem of PostgreSQL, while adding time series capabilities. The source explicitly states, “time scale is a postgres database so it’s still postres SQL”. This also implies that tools and libraries that work with standard PostgreSQL, like SQLModel, can also interact with a TimescaleDB instance.
    • Optimization for Time Series Data: The core reason for using TimescaleDB is its optimization for time series data. Standard PostgreSQL is not inherently designed for the high volume of writes and complex time-based queries that are common in time series applications. TimescaleDB addresses this by introducing concepts like hypertables.
    • Hypertables: A hypertable is a virtual table that is partitioned into many smaller tables called chunks, based on time and optionally other criteria. This partitioning improves query performance, especially for time-based filtering and aggregations, as queries can be directed to only the relevant chunks. We saw the conversion of the EventModel to a hypertable in the tutorial.
    • Time-Based Operations: TimescaleDB provides hyperfunctions that are specifically designed for time series analysis, such as time_bucket for aggregating data over specific time intervals. This allows for efficient calculation of metrics like counts, averages, and other aggregations over time.
    • Data Retention Policies: TimescaleDB allows for the definition of automatic data retention policies, where older, less relevant data can be automatically dropped based on time. This helps in managing storage costs and maintaining performance by keeping the database size manageable.
    • Use in the Analytics API Service: The analytics API service leverages TimescaleDB as its primary data store because it is designed to ingest and analyze time series data (web traffic events).
    • Data Ingestion and Storage: The API receives web traffic data and stores it in the Timescale database. The EventModel was configured to be a hypertable, optimized for this kind of continuous data ingestion.
    • Time-Based Analysis: TimescaleDB’s time_bucket function is used to perform aggregations on the data based on time intervals (e.g., per minute, per hour, per day, per week). This enables the API to provide insights into how web applications are performing over time.
    • Efficient Querying: By using hypertables and time-based indexing, TimescaleDB allows for efficient querying of the large volumes of time series data that the API is expected to handle.
    • Deployment: TimescaleDB can be deployed in different ways, as seen in the context of the tutorial:
    • Local Development with Docker: The tutorial uses Docker to run a containerized version of open-source TimescaleDB for local development. This provides a consistent and isolated environment for testing the API’s database interactions.
    • Cloud Deployment with Timescale Cloud: For production deployment, the tutorial utilizes Timescale Cloud, a managed version of TimescaleDB. This service handles the operational aspects of running TimescaleDB, such as maintenance, backups, and scaling, allowing the developers to focus on building the API. The connection string for Timescale Cloud was configured in the deployment environment on Railway.
    • Integration with FastAPI and SQLModel: Despite its specialized time series features, TimescaleDB remains compatible with PostgreSQL standards, allowing for seamless integration with tools like SQLModel used within the FastAPI application. SQLModel, being built on Pydantic and SQLAlchemy, can define data models that map to tables (including hypertables) in TimescaleDB. A custom Python package, timescale-db-python, was even created to further streamline the interaction between FastAPI, SQLModel, and TimescaleDB, especially for defining hypertables.

    In summary, TimescaleDB is a powerful extension to PostgreSQL that provides the necessary optimizations and functions for efficiently storing, managing, and analyzing time series data, making it a crucial component for building the described analytics API service. Its compatibility with the PostgreSQL ecosystem and various deployment options offer flexibility for both development and production environments.

    Docker Containerization for Application Deployment

    Based on the source “01.pdf” and our conversation history, Docker containerization is a technology used to package an application and all its dependencies (such as libraries, system tools, runtime, and code) into a single, portable unit called a container image. This image can then be used to run identical containers across various environments, ensuring consistency from development to production.

    Here’s a breakdown of Docker containerization as discussed in the source:

    • Purpose and Benefits:
    • Emulating Production Environment: Docker allows developers to locally emulate a deployed production environment very closely. This helps in identifying and resolving environment-specific issues early in the development process.
    • Production Readiness: Building applications with Docker from the beginning makes them production-ready sooner. The container image can be deployed to any system that has a Docker runtime.
    • Database Services: Docker simplifies the process of running database services, such as PostgreSQL and TimescaleDB, on a per-project basis. Instead of installing and configuring databases directly on the local machine, developers can spin up isolated database containers.
    • Isolation: Docker provides another layer of isolation beyond virtual environments for Python packages. Containers encapsulate the entire application environment, including the operating system dependencies. This prevents conflicts between different projects and ensures that each application has the exact environment it needs.
    • Portability: Once an application is containerized, it can be deployed anywhere that has a Docker runtime, whether it’s a developer’s laptop, a cloud server, or a container orchestration platform.
    • Reproducibility: Docker ensures that the application runs in a consistent environment, making deployments more reliable and reproducible.
    • Key Components:
    • Docker Desktop: This is a user interface application that provides tools for working with Docker on local machines (Windows and Mac). It includes the Docker Engine and allows for managing containers and images through a graphical interface.
    • Docker Engine: This is the core of Docker, responsible for building, running, and managing Docker containers.
    • Docker Hub: This is a cloud-based registry service where container images can be stored and shared publicly or privately. The tutorial mentions pulling Python runtime images and TimescaleDB images from Docker Hub. Images are identified by a name and a tag, which typically specifies a version.
    • Dockerfile: This is a text file containing instructions that Docker uses to build a container image. The Dockerfile specifies the base image, commands to install dependencies, the application code to include, and how the application should be run. The tutorial details creating a Dockerfile.web for the FastAPI application, which includes steps like downloading Python, creating a virtual environment, installing requirements, and running the FastAPI application using a boot script.
    • Docker Compose: This is a tool for defining and managing multi-container Docker applications. It uses a compose.yaml file to configure the different services (e.g., the FastAPI application, the database) that make up the application, along with their dependencies, networks, and volumes. The tutorial uses Docker Compose to run both the FastAPI application and the TimescaleDB database during local development. Docker Compose allows for defining build parameters (context and Dockerfile), image names, environment variables, port mappings, and volumes. The –watch flag can be used with Docker Compose to automatically rebuild the container when code changes are detected during development.
    • Container Image: A lightweight, standalone, executable package that includes everything needed to run a piece of software, including the code, runtime, libraries, environment variables, and configuration files.
    • Container: A running instance of a Docker image.
    • Usage in the Analytics API Project:
    • Containerizing the FastAPI Application: The tutorial focuses on creating a Dockerfile.web to containerize the FastAPI microservice. This ensures that the API can be deployed consistently across different environments.
    • Running TimescaleDB: Docker Compose is used to spin up a containerized instance of the open-source TimescaleDB for local development. This simplifies setting up and managing the database dependency.
    • Local Development Environment: Docker Compose, along with features like volume mounting, facilitates a stable local development environment that closely mirrors production. Volume mounting allows for code changes on the host machine to be reflected within the running container without needing to rebuild the image every time.
    • Production Deployment: The containerized FastAPI application is deployed to Railway, a containerized cloud platform. Railway uses the Dockerfile.web to build and run the application.

    In essence, Docker containerization provides a robust and efficient way to develop, package, and deploy the analytics API service and its dependencies, ensuring consistency and portability across different stages of its lifecycle. The source emphasizes using Docker early in the development process to prepare for production deployment.

    Railway Deployment of Analytics API

    Based on the source “01.pdf” and our conversation history, Railway is used as a containerized cloud platform to deploy the analytics API into production. The process involves several key steps to take the Dockerized application and run it in the cloud.

    Here’s a discussion of deployment on Railway based on the provided information:

    • Containerized Deployment: Railway is designed to deploy containerized applications. Since the analytics API is built using Docker, it is well-suited for deployment on Railway. The process leverages the Dockerfile.web created for the application, which contains all the instructions to build the Docker image.
    • railway.json Configuration: A railway.json file is used to configure the deployment on Railway. This file specifies important settings for the build and deployment process, including:
    • build command: This command tells Railway how to build the application. In this case, it points to the Dockerfile.web in the project directory.
    • watchPaths: These specify the directories and files that Railway should monitor for changes. When changes are detected (e.g., in the src directory or requirements.txt), Railway can automatically trigger a redeployment.
    • deploy configuration: This section includes settings related to the running application, such as the health check endpoint (/healthz). Railway uses this endpoint to verify if the application is running correctly.
    • GitHub Integration: The deployment process on Railway starts by linking a GitHub repository containing the application code. The source mentions forking the analytics-api repository from cfsh GitHub into a personal GitHub account. Once the repository is linked to a Railway project, Railway can access the code and use the railway.json and Dockerfile.web to build and deploy the application.
    • Environment Variables: Railway allows for configuring environment variables that the deployed application can access. This is crucial for settings like the database URL to connect to the production Timescale Cloud instance. Instead of hardcoding sensitive information, environment variables provide a secure and configurable way to manage application settings. The tutorial demonstrates adding the DATABASE_URL obtained from Timescale Cloud as a Railway environment variable. It also shows setting the PORT environment variable, which the application uses to listen for incoming requests.
    • Public Endpoints: Railway can automatically generate a public URL for the deployed service, making the API accessible over the internet. This allows external applications or users to interact with the API.
    • Private Networking: Railway also supports private networking, allowing services within the same Railway project to communicate internally without being exposed to the public internet. The tutorial demonstrates how to make the analytics API a private service by deleting its public endpoint and then accessing it from another service (a Jupyter container) within the same Railway project using its private network address. This enhances the security of the analytics API by restricting access.
    • Health Checks: Railway periodically checks the health check endpoint configured in railway.json to ensure the application is running and healthy. If the health check fails, Railway might automatically attempt to restart the application.
    • Automatic Deployments: Railway offers automatic deployments triggered by changes in the linked GitHub repository or updates to environment variables. This streamlines the deployment process, as new versions of the application can be rolled out automatically.

    In summary, deploying the analytics API on Railway involves containerizing the application with Docker, configuring the deployment using railway.json, linking a GitHub repository, setting up environment variables (including the production database URL), and leveraging Railway’s features for public or private networking and health checks. Railway simplifies the process of taking a containerized application and running it in a production-like environment in the cloud.

    FastAPI Python Tutorial: Build an Analytics API from Scratch

    The Original Text

    so apparently data is the new oil and if that’s the case let’s learn how we can control that data and store it oursel and then refine it ourself now what I really mean here is we’re going to be building out an analytics API service so that we can ingest a lot of data we can store a lot of data into our database that changes over time or time series data the way we’re going to be doing this is with fast API as our micros service this is going to be our API inpoint that has well mostly just one data model that we’re going to be ingesting from there we’re going to be using of course python to make that happen that’s what fast API is built in but we’re going to be storing this data into a postgres database that’s optimized for time series called time scale I did partner with them on this course so thanks for that time scale but the idea here is we want to be able to keep track of time series data time scale is optimized postgres for Time series data it’s really great I think you’re going to like it now we’re also going to be using Docker in here to make sure that our application is containerized so we can deploy it anywhere we want and we can use the open-source version of time scale to really hone in exactly what it is that we’re trying to build out before we go into production which absolutely we will be now the idea here is once we have it all containerized we’ll then go ahead and deploy it onto a containerized cloud called Railway which will allow us to have a jupyter notebook server that can connect to our private API inpoint our private analytics server altogether now all of this code is open source and I did say it’s done in fast API which really means that we’re going to be writing some fairly straightforward python functions that will handle all of the API inp points and all that the thing that will start to get a little bit more advanced is when we do the data modeling and specifically when we do the querying on the data modeling now these things I think are really really fascinating but I ease you into them so I go step by step to make sure that all of this is working now I do want to show you a demo as to what we end up building at the end of the day we got a quick glance at it right here but I want to go into it a little bit more in depth now I do hope that you jump around especially if you know this stuff already and if you come from the D Jango world the models that we’re going to be building I think are a little bit easier to work with than Jango models there’s less to remember because they’re based in pantic which is what SQL models is based in as well which allows you to write just really simple models it’s really cool it’s really nice to do so those of you who are from D Jango this part will be very straightforward fast API itself is also very straightforward and a really really popular tool to build all of this out now the nice thing about all of this is all the code is open source so you can always just grab it and run with it and deploy your own analytics API whenever you want to that’s kind of the point so if you have any questions let me know my name is Justin Mitchell I’m going to be taking you through this one step by step and I really encourage you to bounce around if you already know some of these things and if you don’t take your time re-watch sections if you have to some of us have to and that’s totally okay I know I had to repeat these sections many time to get it right so hopefully it’s a good one for you and thanks for watching look forward to seeing you in the course the point of this course is to create an analytics API microservice we’re going to be using fast API so that we can take in a bunch of web traffic data on our other services and then we’ll be able to analyze them as we see fit and we’ll be able to do this with a lot of modern technology postgres and also specifically time scale so we can bucket data together and do aggregations together let’s take a look at what that means now first and foremost at the very end we are going to deploy this into production and then we will have a jupyter notebook server that will actually access our private analytics API service this is completely private as in nobody can access it from the outside world it’s just going to be accessible from the resources we deem should be accessible to it which is what you’re seeing right here once we actually have it deployed internally and private we can send a bunch of fake data which is what’s happening this is pretending to be real web events that will then be sent back to our API as you see here so this is the raw data so from this raw data we will be able to aggregate this data and put it into a bulk form that we can then analyze now of course this is just about doing the API part we’re not going to actually visualize any of this just yet but we will see that if I do a call like duration 2 hours I can see all of the aggregated data for those two hours now this becomes a lot more obvious if we were to do something like a entire month and we can actually see month over month data that’s changing but in our case we don’t have that much data it’s not really about how much data we have it’s much more about how to actually get to the point where we can aggregate the data based off of time based off of Time series that is exactly what time scale does really well and enhances Post cres in that way so if we actually take a look at the code the code itself of course is open source feel free to use it right now you can actually come into the SRC here into the API into events into models you can see the data model that we’re using if you’ve used SQL model before this will actually give you a sense as to what’s going on it’s just slightly different because it’s optimized for time series and time scale which is what time scale model is doing now if you’ve used D Jango before this is a lot like a d Jango model it’s just slightly different by using SQL model and something called pantic don’t don’t worry I go through all of that and you can skip around if you already know these things but the point here is that is the model we are going to be building to ingest a lot of data after we have that model we’re going to be able to aggregate data based off of all of that which is what’s happening here now this is definitely a little bit more advanced of an aggregation than you might do out of the gates but once you actually learn how to do it or once you have the data to do it you will be able to have much more complex aggregations that will allow you to do really complex you know averaging or counting or grouping of your data all of it is going to be done in time series which is I think pretty cool now when we actually get this going you will have a fully production ready API endpoint that you can then start ingesting data from your web services I think it’s pretty exciting let’s go ahead and take a look at all of the different tools we will use to get to this point let’s talk about some of the tools we’re going to use to build out our analytics API first and foremost we’re going to be using the fast API web framework which is written in Python now if you’ve never seen fast API before it’s a really great framework to incrementally add things that you need when you need them so it’s really minimal is the point so there’s not a whole lot of batteries included which gives us all kinds of flexibility and of course makes it fast so if you take a look at the example here you can see how quick we can spin up our own API that’s this right here this right here is not a very powerful API yet but this is a fully functioning one and of course if you know python you can look at these things and say hey those are just a few functions and of course I could do a lot more with that that’s kind of the key here that we’re going to be building on top of now if you have built out websites before or you’ve worked with API Services before you can see how easy it is to also do the URL routing of course we’re going to go into all of this stuff in the very close future now of course we’re going to be using Python and specifically Python 3 the latest version is 3.13 you can use that 3.13 3.14 you could probably even use 3.10 Maybe even 3.8 I’m going to most likely be using 3.12 or 13 but the idea here is we want to use Python because that’s a dependency of fast API no surprises there probably now we’re also going to be using Docker now I use Docker on almost all of my projects for a couple of reasons one Docker locally emulates a deployed production environment it does it really really well and you can actually see this really soon but also we want to build our applications to be production ready as soon as we possibly can Docker really helps make that happen now another reason to use Docker is for database services so if you just want a database to run on a per project basis Docker makes this process really easy and so of course we want to use a production ready database and we also want to use one that works with fast API and then is geared towards analytics lo and behold we’re going to be using time scale time scale is a postgres database so it’s still postres SQL but it’s optimized for time series now yeah I partnered with them on this series but the point here is we’re going to be building out really rich robust time series data and fullon analytics right that’s the point here we want to actually be able to have our own analytics and control everything so initially we’re going to be using Docker or the dockerized open- source version of time scale so that we can get this going then as we start to go into production we’ll probably use more robust service is from time scale directly now we’re also going to be using the cursor code editor this of course is a lot like VSS code I am not going to be using the AI stuff in this one but I use cursor all of the time now it is my daily driver for all of my code so I am going to use that one in this as well now I also created a package for this series the time scale DB python package so that we can actually really easily use fast API and something called SQL model inside of our fast API application with the time series optimized time scale DB that’s kind of the point so that’s another thing that I just created for this series as well as anything in the future that of course is on my private GitHub as far as the coding for entrepreneurs GitHub all of the code for this series everything you’re going to be learning from is going to be on my GitHub as well which you can see both of those things Linked In the description so that’s some of the fundamentals of the tooling that we’re going to be using let’s go ahead and actually start the process of setting up our environment in this section we’re going to set up our local development environment and we wanted to match a production or a deployed environment as closely as possible so to do this we’re going to download install Python 3 we’re going to create a virtual environment for python projects so that they can be isolated from one another in other words python versions don’t conflict with each other on a per project basis then we’re going to go ahead and do our fast API hello world by installing python packages and then of course we’re going to implement Docker desktop and Docker compose which will allow for us to spin up our own database that postgres database that we talked about previously and of course it’s going to be the time scale version The Open Source version of that once we have our database ready then we’ll go ahead and take a look at a Docker file for just our fast API web application so that that can also be added into Docker compose and again being really ready to go into production then we’ll take a look at the docker based fast API hello world now all of this you could skip absolutely but setting up an environment that you can then repeat on other systems whether it’s for development or for production I think is a critical step to make sure that you can actually build something real that you deploy for real and get a lot of value out of it so this section I think is optional for those of you who have gone through all of this before but if you haven’t I’m going to take you through each step and really give you some of the fundamentals of how all of this works just to make sure that we’re all on the same page before we move to a little bit more advanced stuff and advanced features so let’s go ahead and jump in to downloading installing Python 3 right now we’re now going to download install Python and specifically python 3.13 or 3.13 now the way we’re going to do this is from python.org you’re going to go under downloads and you’re going to select the download that shows up for your platform now there are also the platforms shown on the side here so you can always select one of those platforms and look for the exact version that we’re using which in my case it’s Python 3.13 and you can see there’s other versions of that already available that you can download with now this is true on Windows as well but the idea is you want to grab the universal installer for whatever platform you’re using that’s probably going to be the best one for you this isn’t always true for older versions of python but for newer ones this is probably great so we’re going to go ahead and download that one right there and if course if you are on Windows you go into windows and you can see there’s a bunch of different options here so pick the one that’s best for your system now if you want a lot more details for those of you who are windows users consider checking out my course on crossplatform python setup because I go into a lot more detail there now the process though of installing python is is really straightforward you download it like we just did this is true for Windows or Mac and then you open up the installer that you end up downloading and then you just go through the install process now one of the things that is important that it does say a number of times is use the install certificates we’ll do that as well just to make sure that all of the security stuff is in there in as well so I’m going to go ahead and agree to this installation I’m going to go ahead and run it I’m going to put my password in I’m going to do all of those things as you will with installing any sort of software from the internet now I will say there is one other aspect of this that I will do sort of again in the sense that I will install python again using Docker and specifically in Docker Hub so yeah there’s a lot of different ways on how you can install Python and use it but both of these ways are fairly straightforward okay so it actually finished installing as we see here and it also opened up the finder window for me for that version if you’re on Windows it may open this folder it might not I haven’t done it in a little while but the idea is you want to make sure that you do run this installation command here which if you look at it is really just running this pip install command we’ll see pip installing in just a moment but that’s actually pretty cool so we’ve got pip install uh you know the certificate you know command basically to make sure all that security is in there now once you install it we just want to verify that it’s installed by opening up another terminal window here and run something like Python 3 – capital V now if you see a different version of python here there’s a good chance that you have another version already installed so for example I have python 3.12 installed as well you can use many different versions of python itself all of these different versions are exactly why we use something called a virtual environment to isolate python packages again if you want a lot more detail on this one go through that course it goes into a lot more detail on all different kinds of platforms so consider that if you like otherwise let’s go ahead and start the process of creating a virtual environment on our local machine with what we’ve got right here part of the reason I showed you the different versions of python was really to highlight the fact that versions matter and they might make a big issue for you if you don’t isolate them correctly so in the case of version 3.12 versus 3.13 of python there’s not going to be that much changes in terms of your code but what might have a major change is the thirdparty packages that go in there like what if fast API decides to drop support for python 3.12 and then you can’t use that anymore that’s kind of the idea here and so we need to create a virtual environment to make that happen which is exactly what we talked about before so back into cursor we’re going to go ahead and open up a new window inside of this window we’re going to go ahead and open up a new project so I want to store my projects in here so I open up a new project I find a place on my local machine as to where I’m going to store it and we’re going to call this the analytics API just like that and I’ll go ahead and open up this folder okay so normally with cursor it’s going to open you up to the agent I’m not going to use the agent right now I’m just going to be using just standard code and we’re going to go ahead and first off save the workspace as the analytics API we’ll save it just like that then I’m going to go ahead and open up the terminal window which you can do by going to the drop down here or if you learn the shortcut which I recommend you do it can toggle it just like I’m doing okay so the idea here is we want to just of course verify that we have got Python 3 in there you can probably even see where that version of Python 3 is stored this actually shouldn’t be that different than what you may have saw when we installed called the certificates here but the idea of course is we’re going to use this to activate our virtual environment so I’m going to go ahead and do python 3.12 or rather python 3.3 13 and then do m venv v EnV so this is the Mac command for it this also might work on Linux if you’re on Windows it’s going to be slightly different which is going to be basically the absolute path to where your python executable is so it’s going to be something like that then- MV andv V andv okay so this is another place where if you don’t know this one super well then definitely check out the course that I have on it uh which was this one right here so just go ahead and do that okay so the idea is now that we’ve got this virtual environment all we need to do is activate it now the nice thing about modern text editors or modern code editors is usually when you open them up they might actually activate the virtual environment for you in my case it’s not activated so the way you do it is just by doing Source VMV and then b/ activate this is going to be true on Linux as well Windows is going to be slightly different unless you’re using WSL or you’re using Powershell uh those things might have a little different but more than likely Windows is going to be something like this where it’s uh VMV scripts activate like that where you put a period at the beginning that will help activate that virtual environment and so now what we do is we can actually do python DV notice the three is not on there and here it is and of course if I do which python it will now show me where it’s located which of course is my virtual environment and so this is the time where we can do something like python DM pip or just simply pip both of those are the python package installer and we can do pip install pip D- upgrade this is going to happen just on my local virtual environment it does not affect pip on my local machine which we can check by doing pip again if I do that notice that it’s not working right so it works in here where I do pip but it does not work in my terminal window nonactivated terminal window if I do python 3-m pip um then I’ll get it or rather just pip that will give me that actual thing and it’s showing me where it’s being used just like that same thing if I came back in to my virtual environment scrolled up a little bit it will show me something very similar here’s that usage if I do the python version it will show me the same sort of thing that we just saw but it’s going to be based off of the virtual environment just like that so that’s the absolute path to it of course it’s going to be a little bit different on your machine unless you have the same username as I do and you stored it in the same location but overall we are now in a place to use this virtual environment so let’s see how we can install some packages and kind of the best approach to do that now let’s install some pyth packages it’s really simple it’s a matter of pip install and then the package name that is how the python package index works if you’re familiar with something like mpm these are very similar tools but the idea here is if you go to p.org you can search all of the different P published python packages and pip install can install them directly from there so if we did a quick search for fast API for example we can see here is the current version of fast API that’s available and this is how we can install it now the key thing about these installations is just like many things there’s many different ways on how you can do this there are other tools out there like poetry is another tool that can do something like poetry ad I believe that’s the command for it but the idea here is there’s a lot of different ways on how you might use these different package names I stick with the built-in modules cuz they are the most reliable for the vast majority of us now once you get a little bit more advanced you might change how you do virtual environments and you also might change how you install python packages but the actual python you know like official repository for all of these different packages is piie and they still say pip install so it’s still very much uh a big part of what we do okay so the idea here now is we need to install some packages so how we’re going to do this is we’re going to go ahead and once again I’m going to open up my project and in my case I actually closed it out mostly so I can show you the correct way to install things here’s my recent project here with that workspace all I have is a virtual environment and that code workspace in there now if I were to toggle open the terminal it may activate the virtual environment it may not so the wrong way to do this is to just start trying to do pip install fast API in this case it says commands not found the reason this is the wrong way is cuz I don’t have the virtual environment activated so I have to activate that virtual environment just like that if I need to manually do it if for some reason the terminal is not automatically doing it then we can do the installation so we can go ahead and do pip install fast API and hit enter and just go off of that and that’s actually well and good except for the fact that I don’t have any reference inside of my project to the fast API package itself so I have no way to like if I were to accidentally delete the virtual environment I have no way to like recoup what I did so what we need to do then is we create something called requirements.txt once again this is another file that could be done in different ways and there are other official ways to do it as well but one of the things that’s nice about this is inside of this file we can write something like Fast API and then I can do pip install d r requirements.txt which will then take the reference from this file assuming that everything’s installed and saved and so once I save it I can see that that’s the case it’s now doing it this comes in with the versions then so what we do then is we can actually grab the version and say it’s equal to that version right there in which case I can run the installations now this is actually really nice because it kind of locks that version in place now if you go into the release history you could probably go back in time and grab a different version like 011 uh 3.0 if we do that so 0113 and then 0.0 I save that now I run that installation it’s going to take that old one and it’s going to install the things that are relative to that uh inside of my entire environment now in my case I actually want to use this version right here and a lot of times I would actually do another tool to make sure that this version is correct that other tool is called pip tools which I’m not going to cover right now but the idea here is we want to keep track of our requirements and fast API of course is one of them now within fast API we have something else called uvicorn that we will want to use as well this is often hand inand with using fast API so once again we see pip install uvicorn and you can just do something like that now there is something else with uvicorn that we might want to use and that’s called gunicorn so G unicorn and we do a quick search for that this is for when we want to go into production G unicorn and uvicorn can work together so once again I’ll go ahead and bring that in uvicorn and gunicorn they are um great tools for running the application in production but we usually don’t have to lock down their version I think once you go into production you need regular things then yeah you’ll probably want to lock down the version the more important one is probably fast API but even that we might not need to lock down our version at this stage like if you’re getting something from what I’m telling you right now then you probably don’t need to lock down the version yet if you already know oh I need to lock down the version then you’ll probably just do it anyway that’s kind of the point that’s why I’m telling you that um but the idea here is of course we’ve got all these different packages and then of course my package that is definitely a lot newer you do a quick search for it and it’s time scale DB it is not a very like there’s not that many versions of this package so this is another one that I’m going to go ahead and not lock down because I definitely don’t want that so within that package there is something called SQL model which is a dependency of the other package so it’s this one right here this one also doesn’t have that many versions itself uh but it’s pretty stable as it is and this one works well with fast API and you’ll notice that it’s powered by pantic and SQL Alchemy which means that we probably want to have those in there as well just so we have some reference to it so I’m going to go ahead and bring that in here with pantic and SQL Alchemy there we go so now that we’ve got all of the different versions here I can once again do pip install R requirements.txt and then I’ll be able to install all of the packages as need be now here’s the kicker this is why I’m showing you is because what you want to think of your virtual environment as just for the local environment you are not going to be using it in the production environment you’ll be doing a completely different one what’s interesting is my terminal now opens up that virtual environment and attempts to activate it which is pretty funny but I don’t actually have one yet so I’m going to go ahead and re bring it back now I’ve got that virtual environment back of course if you’re on Windows you might have to use something different here but I’m going to go ahead and reactivate it with bin activate and then we’ll go ahead and do PIV install r requirements. txt hit enter and now I’m getting all of those requirements back for this entire project this is going to come up again we will see it for sure because it’s really important okay so now we’ve got our requirements we’ve got our virtual environment it’s time to actually do our fast API hello world before we go much further I will say if there are file changes within the code it will be on the official GitHub repo for this entire series as to what we’re doing going forward so in the in other words in branches of start this is the branch we’re going to finish off with of course this one doesn’t have everything in it like we just did but it will and so that’s the start Branch that’s where we’re leaving off but as you see there’s license and read me those things I’m not going to show you how to do or the get ignore file uh they’re just going to show up in just a moment but now that we’ve got this let’s go ahead and actually create our first fast API application it is very straightforward we now want to do our fast API hello world so the way we’re going to do this is by creating a python module with some fast API code and just run it the key part of this is just to really verify that our package runs that it can actually go so to do this we’re going to go ahead and go back into our project here and of course I want to open up my terminal window if the virtual environment activates that’s great if it does not activate like that then we want to activate it manually every once in a while when you create a virtual environment you might see this where it automatically activates we hope that it will start to automatically activate I’ll mention as to when it might happen but the idea here is we want to activate that virtual environment so Source VMV bin activate or of course whatever you have on your machine then I’m going to go ahead and do pip install R requirements.txt it’s not going to hurt to run that over and over and over again right it never will because if it’s already installed it’s not going to do anything cool other than tell you that it’s already installed so the idea here is we want to make sure that it’s installed so we can actually run it the next question of course how do we run it well if we go into the fast API documentation and we go to the example we see here some example python code so it says create a file main.py with all this stuff so the question then is where do we actually put this and that’s going to be as simple as you could do it right here in main.py you can copy this code paste in here save it with command s which is exactly what I do a lot I won’t actually show you that I’m saving it I’ll just save it I just saved it like 10 times just there so now that we’ve got that we actually want to run this code right so how do we actually run it well going back into the docs again you scroll down a little bit you see that it says run it right here and so I’m going to go ahead and attempt to run it with fast API Dev main.py hit enter I get an error so there’s a couple things that I want to fix before this goes any further but the idea here is this may might make you think think that oh we didn’t install fast API correctly and in a way we didn’t we didn’t install it to use this particular command line tool it is a pce ma command line tool it’s not necessarily going to be there by default in this case it’s simply not there now in my case you could I could consider using this as my development server for fast API what I actually want to use is uvicorn because it’s closer to what I would use in production and it’s actually what fast API is using anyway as you look in the documentation you can scroll down a little bit it says we’ll watch for changes uvicorn running and all that so it’s basically uvicorn anyway so that’s what we want to do is we want to have uvicorn run our our actual main app so you use uvicorn then this is the command if you hit enter you should be able to see this now if you’re on Windows you might actually have to use waitress at this point waitress is just like uvicorn but that might be the one that you end up using we’ll talk about solving that problem in just a little bit when we go into dock but for now um I’m going to be using uvicorn and of course if you are on Windows go ahead and try that fast API standard that will probably work for you as well okay so the idea here is now we want to actually run this application with uvicorn and then the way we do that is we grab main.py here and then we use colon looking back into main.py we look for the app declaration for fast API which is just simply app and then we can do something like D- reload hdden enter oh that trailing slash should not be there so I’ll try that again we hit enter and now once again it says we’ll watch for these changes and it’s running on this particular Port just like what we see right in here great it just doesn’t show us this serving at and docs at and stuff like that as well is there’s not a production version of this just yet that we’ll use we will see it though okay great so now I can open this up with command click or just copy this URL you can always just control click and then open it up like that but the idea is hello world it’s running congratulations maybe this is the first time you’ve ever run a fast API application and it is up and running and it’s working on your local machine now don’t get too excited because you can’t exactly send this to anybody right so if I were to close out this server which we have a few options to do so one is just doing contrl C which will do that it just literally shuts it down another option is just to you know kill the terminal and then you can open up the terminal again in which case ours right now we need to then reactivate it and then we can run that command Again by just pressing up a few times we’ll be able to find it and there it is okay great so now we have it running what can we do we well we need to actually put it in a location that makes more sense than where it is right now main.py is in the root folder here this is basically something that no one ever does when you start building you know professionally you often put it inside of SRC main. P like that notice that I put that that slash in there that’s important because as soon as I hit enter it actually creates a folder for me with that new module which is where I’m going to go ahead and actually paste all of that code again and then I’ll just go ahead and delete this one with command backspace that allows me to delete it of course there’s other ways to delete it but there we go okay so now we can verify that fast API is running it’s working it’s ready to go so if you’re on Windows and you have Docker installed or if you just have Docker installed the next part is going to allow for us to build on top of this or the next half of this section is going to help us build on top of this so that we can actually use this code a little bit more reliably than we currently do let’s say you want to share this application with one of your friends and you want to help them set it up to run it what you’d probably tell them is hey download python from python.org make sure that you create a virtual environment install these requirements and then use uvicorn to run main.py and then you remember oh wait you also have to activate the virtual environment then install the requirements then run with u viacor then you might remember oh make sure you download python 3.13 because that’s the version we’re going to be using so there is some Nuance in just the setup process to make sure that this is working correctly that of course you and your friend could figure out by talking but it would be better if you could just give them something and it just worked docker containers are that something what we end up doing is we package up this application into something called a container image and we do it in a way very similar to what we’ve been doing which is download install the runtime that you want to use like Python 3 create a virtual environment activate that virtual environment install the requirements and then run your P your you know your application your fast API application so we will get to the point where we actually build out these things very soon but right now I want to just show you how you can run them how you can just use them by downloading Docker desktop so if you go to dock. and hit download Docker desktop for your platform you’ll be able to get the docker desktop user interface just like this but you will also more importantly get Docker engine to be actually able to run Docker containers to be able to create them and to be able to share them the key parts of using Docker now Docker itself can be really complex so I want to keep things as straightforward as possible so it’s very similar to what we did with python when we downloaded and installed it but we’re going to do that the docker way right now so at the point that you download it you’re going to go ahead and open it up on your machine and it’s going to look something like this UI right here now if for some reason this isn’t popping up and it’s just in your menu bar you might actually see that as soon as you exit out it might be something like this in which case you’ll just go to the dashboard and that will actually open it back up that’s just off of the recording screen for me right now which is why you’re seeing it this way but the idea here is we now have the docker desktop on our local machine now I also want to verify that this is working correctly because if I don’t verify it then it’s going to be hard to run in general so opening up the terminal I should be able to do Docker PS Docker PS really just shows me all of the things that are running on my machine we’ll come back to this in just a moment but the idea is if this runs an error you don’t have it done correctly the error will look something like that so just type in gibberish that’s the error right command not found in this case so I want to make sure that that’s there and I also want to make sure that Docker compose is there we’ll come back to Docker compose really soon but for now we’re just going to have those two commands now I’m going to be giving you a very minimal version of using Docker and learning how to use Docker over the next few videos or next few parts of this section now the reason for that has to do with the fact that we need it for a very stable local development environment but also so we can put push this into production so let’s actually take a look at how we can actually use a Docker container image right now so on Docker desktop there’s this Docker Hub right here there’s also Docker Hub if you go to hub. deer.com both of these things are just simply Docker Hub and you can search for other container images it’s not surprising that if you look at the docker Hub on Docker desktop versus the website they look basically the same that’s of course makes a whole lot of sense now in our case we’re going to go ahead and do a quick search for python we want to do the python runtime so when we went to python.org we went to downloads and you could have gone to one of your platforms here and you could actually grab a specific version of python so this is true on dockerhub as well but instead of saying download it’s just a tag so you just go into tags here and you can find different versions of python itself you can get the latest version which often is a good idea but just like what we talked about with the actual versions of you know packages or python software that we’re using a lot of times you want to use a very specific version to make sure that all of these things work as well so going back into uh you know the dockerhub we’ll do a quick search for Python 3.13 and so what we see in here is python 3.13 the one we’ve been using here’s 3.1 3.2 and of course if we go back into you know actual python releases we can see there’s 3.12 right there and and it’s the same one exact same one is through Docker one is directly on your local machine so the one through Docker is going to work on Linux Windows Mac it’s going to work crossplatform it’s really great that it does that but I actually don’t want to use the version that I already have on my machine I want to use an a really old version so Python 3.16 and we’ve got relevant and it has vulnerabilities there’s a lot of security concerns with using this old one for our production systems but for this example there’s no con concerns at all we can delete it it’s really simple to do so the way we actually get this thing working is by copying one of these pole commands so we can scroll on down I’m going to go ahead and just grab the one that is 3.6.1 I’m going to go ahead and copy this command here go into my terminal and I’ll go ahead and paste that in with that pole command so this is going to go ahead and download python 3.1 or 3.6.1 5 directly from dockerhub it downloads it in the image or the container form so we can now run this with Docker run python Colin 3. 6.15 and hit enter and it immediately stops so the reason it immediately stops is because Docker itself has a lot of features it can do a lot of things one of those things is we can do an interactive terminal for this which is- it with that same Docker container image and we can hit enter there now it actually opens up python for me it’s quite literally in the python shell much like if we were to open up a new terminal window here and type out Python 3 in there that is also a python shell right the difference here is if we actually clear this out I can just run it at any time and I can change the version there’s also another difference that says Linux up here here and Darwin down here that’s because on my local machine I’m on on a Mac but in the docker container I’m on a Linux so Docker containers are basically Linux with a bunch of things already set up now the other cool thing about this is I can exit out of here of course and then I can run python 3.7 and now it says it can’t find it so it’s going to go ahead and download it for me all the while on my local machine if I try to do python 3.7 it’s not available python 3.6 not available very similar to using a virtual environment but it’s another layer of isolation and it allows us to have these packaged run times that we can use at any time which is fantastic so the other part about this of course is that inside of Docker desktop we can delete these old ones that we don’t want to use and you’re going to want to get in this habit sometimes to make sure that you don’t have these old versions of python on your machine because they’re rather large that’s why there’s other versions or other tags of it right so if you go back into Docker Hub you’ll see that there’s this slim Buster look how much smaller of a an actual python project it is it’s way smaller so you can just grab that version and then you can come in here and do the same thing Docker run-it that version right there this is going to download it it’s going to be much smaller which means that it probably doesn’t have nearly as many features out of the gates it’s still on Linux but we can do you know all sorts of python things in here as you would and then you can then see inside of Docker desktop all of the things that are downloaded in here and then the slim Buster is quite a bit smaller than these other ones and so another thing about Docker desktop that’s really nice is you can come in here you can search for Python and you could do a quick search which gives us all of these ones that are using the python image all of these are running versions of the application which you can stop and delete which you would want to do and then you go into your images you can also delete these as well just like that which gives you back 2 gigs of space which you’re definitely going to want to get in the habit of doing in which case then when you want to run it again in the future it will just go ahead and redownload that image right so it’s really meant to be ephemeral like this you’re meant to think of Docker as something that is temporary so you can add it remove it you know run it when you need to and then delete it when you don’t right and so every once in a while you’ll see hey you can’t delete it because it’s running so then that means you just go into the containers here these are ones that are running you just go ahead and delete that you go back into the images and then you can delete it worst case scenario you just literally shut down Docker desktop open it back up and you can see what’s running with Docker containers in here or you can use Docker PS to see if anything’s running right now I don’t have anything running which is very clear in the desktop as well as my terminal okay so we’ll continue to use this I don’t expect you to know all of Docker at this point you shouldn’t know all of Docker at this point instead you should just be able to benefit from using Docker which will allow for us to do do all sorts of Rapid iteration and isolate our projects from each other on a whole another level well beyond what virtual environments can do and it will allow us to have as many database instances as we might want which is what we actually want to do very soon by using time scale itself so that is the docker desktop and some things about Docker compos now I actually want to get fast API ready we want to actually build our own container for fast API or at least the makings of it then we’ll go ahead and take a look at how we can develop with Docker itself we are now going to bundle our fast API application as a container the way we do this is by writing some instructions in a specific syntax so that Docker can build the container from those instructions and our code so this is called a Docker file so we’ve got a production Docker file for fast API the reason we’re doing it now is because the sooner we can have our project optimized for production the sooner we can actually go into production and share this with the world or anyone who has a Docker runtime those things are the key part of this so the actual Docker file we’re going to be using is on my blog so it’s here this is a production version that you can just go ahead grab and run I’m going to make a modification to it for this project but that’s the general idea here here so I’m going to open up my project now I’m actually going to copy these steps you don’t have to copy them but I just want to lay them out so I know where I want to go with this instruction this Docker file so if I go into Docker file like this no extension I’m just going to go ahead and paste this out and these are the instructions I wanted to do first off is download Python 3 create a virtual environment install my packages and then run the fast API application those four steps are really what I want to have happen now the way this works is very similar to like when we ran our python application itself right when we ran it just with Docker itself we can see that it is in Linux and here is our docka application so it really starts at this tag here the container and its tag so the way we find this of course is by going into Docker Hub looking for a container image like we did looking for the version that we want to use and then also the tag that we want to use which of course is going to be 3 uh 13. 2 which we had locally that’s the one we downloaded in the first place we’re going to use that same one for our Docker container so the big difference here though is we don’t want to use the big one because it’s massive it’s massive all across the board we want to use a small one which is called slim Bullseye so that’s the one we actually want to use and so the idea here is very similar to what we have with the slim Buster from the original python download in the docker container we’re going to do something very similar to this the way it works is we say from this is the docker um you know Syntax for it the actual image that we want to use which we could use latest this might be 3.13 this might be 3.14 this might be 3.15 we want to be very specific about the version we’re going to use which again 3.13 and then I think it was2 is the Baseline one if you do slim Bullseye slim blls eye like that it will be a much smaller image as soon as you do a much smaller image we lose some of the default things that might come within Linux itself when we lose that that means we need to also install our Linux environment so the next step might be you know set up Linux OS packages right so if you were going to deploy this directly on a Linux virtual machine you would need to do that same idea that same concept here now I could go through all all of these steps with you or we can just jump into the blog post and copy it because this is not a course about Docker so I’m going to go ahead and copy what’s in this blog post and we’ll go ahead and bring it right underneath those comments and I’m going to go ahead and modify this a little bit to make it work for our project the first thing is notice that I’ve got this argument in here we actually don’t need that argument we’re just going to go and stick with that single one right there and then I’ll just go line by line and kind of explain what’s going on first off we create that virtual environment no big deal this time it’s going to be in a specific location this is like as if we were setting up a remote server we would want our virtual environment in one location because that one server is probably going to have only one virtual environment for this particular application which is what we’re doing with our container this one container is going to only have one python project but we still want to use the virtual environments to isolate any python that might be on the operating system itself so now that we’ve got this virtual environment here this little Command right here makes it easy so that we don’t have to activate it each time it’s just activated and so we can run the PIP install command and upgrade pip we do python related stuff here’s the OS dependencies for our mini virtual machine or our Mini Server here we can then install things like for postres or if you’re using something like numpy you would have other installations in here as well or if you wanted to have git in here you could install that as well so a lot of different things that you can do on the OS level and that’s it right there so that’s one of the cool things about using Docker containers themselves is you also control the operating system not just the code so what we see now is we are making a directory in this operating system this mini one called code we have a working directory in here also called code in other words we are just going to be working inside of there then we copy our requirements file hey what do you know requirements.txt into an absolute location this doesn’t matter we could actually not have it in an absolute location but it is nice that it is in one because later when we need to install install it that’s what we do and so what we see here is it copies the code into the container’s working directory in other words it’s copying main.py into a folder called code no big deal now we’ve got a few other things that we probably don’t need in here um and then the final one is actually running a runtime script A bash script to actually run this application and then removing some of the old files to reduce image size and then finally running the actual script itself so what I want to do before I say this is done is I actually want to create something called boot slocker run. SH now the reason I’m doing this is because all of us are going to need to know what it is that’s going on with our application at any given time and this is what it’s going to be so we first off make this a sh file or bin bash file so that the Linux virtual machine will be able to run it the Linux container will be able to run it then we want to CD into the code folder we also want to activate the virtual envir M then we have runtime and variables to actually run our application which is going to be an SRC main app and that is a little bit different than what we’ve got here so I’m going to go ahead and just keep it in as main app just for now we might change this in the future but this is going to be our script to actually run our application and so to use that script what we do then is we are going to go ahead and copy theboot slocker run. sh and we’re going to go ahead and copy that that into opt runsh so instead of this script here we’ve got a new one and that’s going to be the name of it then we want to do the Cho mod to make it executable then that’s what we’re going to end up running is that script right there and make sure it has a.sh and that’s it so of course I still need to test and make sure that this is working it’s not necessarily working already so I will have to make some changes in here just to make sure that that’s the case that’s not something we’re going to worry about yet the main thing here is that we have a Docker file and that we’re going to be able to build it really really soon this Docker file most likely won’t change that much if anything the blog post will give the updates to the change or the actual code itself will have this Docker file in here that’s the key of Docker files they don’t need to change that much the only thing that will change most likely would be the python version that you end up using over time and then the code that’s going in but everything else related to this is probably going to remain pretty static that’s also why in the blog post the way we run the actual application itself is written right in line it’s actually a script that’s created right in line but having it external makes it go a little bit faster than what we have right here now I realize some of you aren’t fully ready to learn the inss and outs of Docker but we want to be as close to production as possible which is why that blog post is exists and it’s why you can also just copy this stuff personally I think looking at this it’s hopefully very clear as to the steps that need to happen to recreate our environment many many many times over so that it’s a lot easier to share it whether it’s with somebody else who has a Docker runtime or with a production system over the next few parts we’re going to be implementing the docker compose based fast API hello world but before we get there we need to still see some things about Docker just so you have some Foundation as to what’s happening in Docker compose so jumping back into our project here we’ve got this Docker file these are the instructions to set up the tiny little virtual machine or the tiny little server to run our application right it bundles everything up on our code so we need to actually be able to build out our application so there’s really two commands for this I’m going to put them in our read me here so we’ll go ahead and do Docker in here and we’ll go ahead and do the two commands and it’s Docker build and then it’s going to be Docker Run Okay so build is what it sounds like it’s going to build our bundled container image and the way we do it is we tag it something hey these tags what does that remind you of hopefully it reminds you of a few things a tag like this and in our terminal a tag like this right so we need to tag it in the same way it’s going to be tagged on our account if we were pushing it into dockerhub which we’re not but we still need to tag it so either way we’re going to build it and tag it then we want to say hey where are we building this file well we’re going to build it in the local folder which is just period there now we also want to specify the docker file we’re going to use which is how we do it like that now if you don’t specify the docker file it’s just going to go based off of that as in no other Docker file the reason being is you can have multiple Docker files like docker file. web in which case you would want to specify something like Docker file. web we aren’t doing that we’re just using the one single Docker file but it’s important to know about if you were going that direction on how you go about building it once you build it then you run it hey what do you know build it then run now we’ve already seen that command a little bit as well too which was Docker run and then that python command like this now the thing about this run command is there are a bunch of arguments that can go in here there can be the- it argument like we saw with python we ar going to spend any time with the arguments in here in fact this is all we want to see for our arguments at this point because we actually want to use Docker compose because it will do all of this for us as we’ll come to see so the idea here is we want to build out the container let’s go ahead and do that in the root of our project right next to Docker the docker file itself I’m going to go ahead and run this command and it’s going to go ahead and build this out for me now in my case it actually went really fast cuz there’s this cache in here I actually did test this and built it already so if I go into Docker desktop I can see the image that was built in here which will be my image my actual Docker analytics app image in here so I have got a few of them in here and the reason that I have a few is because I was testing this out but of course you can delete these just like we’ve seen before in this case we’ve got one that I can’t actually delete yet so go back into my containers in here and let’s go ahead and get rid of the search bar and I’m going to go ahead and stop all of my containers and delete all of them and then I’ll go back into my image here and I’m going to delete this one as well just so I can see it being built out I also wanted to show you that’s what you do if you want to get rid of it whether it’s your app or someone else’s and so once again it still has cash in here so it’s going really really fast which is super nice but sometimes it will take a little bit longer than that now there’s this other Legacy warning that we’ve got in here CU one of the docker files needs to be changed a little bit to having equals instead of that space bar there so I’m going to go ahead and do that and I’m going to try and build it again cool so it builds really fast great so now I’ve got a container image that I can run it’s not one that’s public it’s only on my local machine it goes public when I push it into dockerhub which like I said we’re not going to do at this point so to run it I just go ahead and do Docker run and then whatever that tag is in our case that tag is anal analytics API which we can also verify inside of our images in here there’s the the tag itself well actually it’s the name of the container with latest in here so it actually def defaults to latest that’s the tag that’s how you actually can do it it’s going to default to that if you don’t specify so if you were to specify one it would just be like that again we’ll do some of this stuff with Docker compose in just a little bit so now all I really want to do is verify that I can run this application by doing Docker run and there it is it’s now running the application itself if I try to open this application it’s not going to work we will make it work in a little bit the reason it’s not working is because of how Docker Works itself everything needs to be explicit to make it work so in order for it to run on a specific Port we also have to let Docker know about that the application itself doesn’t have to let Docker know about that we’ve got a lot of control over how all of that works so what I want to see now is how to do these two things inside of Docker compost off the video I went ahead and stopped that container and deleted the built image so that I could then run the command Docker run analytics API which of course failed it’s not locally and it’s also not on Docker Hub so it just can’t run it so that’s a little bit of an issue that actually is overcome by Docker compose so if we go in here and do compose diamel we can actually start specifying the various Services we might need so the very first key in this yaml is going to be services in inside of there are going to be all of the container images we might want to use like a database or our app in this case we’ll go ahead and just work off of our app the nice thing about modern tooling is a lot of times you can just run Individual Services right inside of the ammo file it also might depend on an extension that I have installed but the point here is we want to make sure that we have these nested key value pairs Here app is just what I’m calling it I could call it SRC I could call it web app I could call it a whole lot of things what we call it is going to make more sense or it’s going to matter more later when we use something a little bit different than just app okay the idea here in then is we’ve got our app and now we want to specify the image name so what do we want to call it well we could call it the same image name as in analytics API and this time we can say something like V1 okay great next up what we can do is Define the build parameters here and that is going to be our context which is going to be the local folder this should remind you of this dot right here so that’s the context that we’re looking at and then the next part is specifying the docker file we’re going to use relative to the composed file so we’ll set Docker file and it’s going to be Docker file just like that now if we had this as Docker file. web which you might do at some point then you would just change this to web as well let’s actually keep it like that so it’s a little bit more clear as to what’s going on in here now if we do change it one note I will change is inside of my little command here I would want to change that one as well just in case I wanted to build it individually okay so the idea now is I’ve got my Docker image and some build parameters now you can actually add additional build parameters in here this is all we’re going to leave though is the context and the docker file in part because it actually matches what we did to build it in the first place but the other part is well we probably don’t need a whole lot of context just to build it because that’s what the docker file is for the docker file has a lot of those context things that you might need to build it now to run it it’s a whole another story so to run it what we did was well actually if I just leave it like this and try to run it we’ll see something interesting so I’m going to go ahead and clear this out and then do Docker compose up hit enter what this will do is it starts to run it but it actually ends up building the application itself and then it goes to run it which it is now running and of course if I were to go to this actual place it will still not work right okay so we’ll get to that in a second but it built and ran it basically the same way but now we specify the image here and so the command I need to remember is just simply Docker compose up and if I want to take it down I can open up another terminal window and do Docker compose down that will stop that container application from running which I think is pretty nice okay and then we could also go into Docker desktop we can take a look at the images in here and look for our analytics API and what do you know there’s that V1 tag in there as well uh which makes things a little bit nicer and easier to see what’s going on so the image was built with that tag is kind of the point okay great so now what we want to do though is we want to be able to actually access this inpoint this URL here so the way it works with Docker this is true whether it’s Docker compos or just straight Docker itself as in this run command we actually need a specify ports so part of the reason that I actually had ports in the first place like inside of my Docker run is so that I can specify a different port right so this port value this environment variable we want to play around with this in just a moment so the way we play around with this is going to happen uh next to Ports so before I actually do the ports let’s go ahead and change the environment variable so we’re going to go ahead and come in here and not to use entry point but

    yeah the rather use environment and set key value pairs so the port value you want to use so let’s go ahead and use port value 8,000 and2 I’m not going to do the ports just yet we’ll just change the port and then I’ll go ahead and do Docker compose up notice that the environment variables have changed if I try to open it once again it still is not accessible okay so this is in part because what I changed was runtime arguments this little thing changed a pretty big change inside of the application because of this environment variable or more specifically this one right here so inside of our composed. gaml we’ve got this port value in here now the reason I’m mentioning this is because at some point we will have a database URL in here and we’ll be able to pass in what that argument is another thing that we can do inside of using you know just Docker compose is do an EnV file and you can do something more like EnV and which case you could just come in here and Dov and this being Port like 801 so right now I have conflicting environment variable values so let’s actually see what that looks like I’m going to go ahead and call Docker compose down and then we’ll go ahead and bring it back up in just a second okay took a few seconds for that to finish but now if I do Docker compose up it’s still Port 802 in other words these hardcoded environment varibles override the environment variable file so if I were to get rid of that Port it would still be uh you know whatever the environment variable value is uh so for now I’ll just leave them the same so there’s not any confusion but the idea here is we have the ability to have environment variable files now we can add more of them by just using another line here you can do something like em. sample or any other kind of environment variable files and of course it has to be in this format of key equals some value okay great so the final step here is really just to expose this port so I’m going to go ahead and bring this down and the way this works is we declare ports in here and our this is super cool so what it shows us is our host port and our container Port so if I click on that we’ve got host Port of 8080 goes to container Port of 80 if you’re not familiar with what the container Port is that is going to be this number right so Port 802 and then the host Port is our system what port do we want to access Port 802 on this so this sometimes doesn’t work as intended so we’re going to go ahead and try it out we’ve got Docker composed up here’s 802 and then we exposed Port 8000 so inside of my Local Host here I’m going to try to do port 8080 and see if I can connect so depending on how your system ends up being designed this might work and it also might not work uh so what you want to typically do is default to your local host port to that Port itself the reason for this has everything to do with how our port and host and all this sort of stuff is being mapped it sometimes works very seamlessly sometimes does not basically the general rule of thumb is if you have the ability to use the same port you should but the reason I wanted to show you all these different ports here is because this is kind of confusing it doesn’t really show you what’s going on you just need to remember the first Port is going to be our Systems Port the second Port is going to be the port that the docker container app is running on great so now we can go ahead and bring that back down and then we’ll go ahead and bring it up in just a second okay let’s bring it back up and here we go so we got Port 802 we open this up and now we’ve got the hello world in there so congratulations you have Docker compose working now this stuff I realize might be a little complicated if you’ve never done this before but really these are just a bunch of arguments that we are telling docker to use for this particular container image that’s being built locally you don’t always build things locally as we saw before when we actually had a python project but when we do build things locally we have a whole another set of stuff that we can do so for example we can change the docker file command here the one that starts the application the runtime script we can change it to something different something like uvicorn main app and then the host port and reload the nice thing about this then is it’s actually going to be built off of that command it’s not going to be using the gunicorn script at all so if you made any mistakes with that you could just use this command to run it and of course you would take it down and make it all work just like that um as it need be so the next thing about the docker compost stuff is you can do something like this where you can actually watch for changes on files and it will rebuild the entire container we’ll see that in just a moment now if you wanted to rebuild the container or if you want to be able to do your own development you’re going to do one more thing which is attaching a volume to this container so this volume here is going to go ahead and grab this folder of source and it’s going to go ahead and put it where we want it to right now this says slash apppp which may or may not be the correct location we want to mount source so this means that we go back into the docker compost file and we scroll down and we see where we copy our SRC folder which actually goes into slash code so we just make a minor modification to this volume here and that means that we are going to go ahead and basically copy our code into the Container constantly and then we’ll be able to rebuild it so this might be confusing so let’s go ahead and see what what I mean by all of this we do Docker compose up d-at so what’s happening here is we see that watch is enabled if I were to change something to requirements.txt let’s go ahead and say bring in requests here like python requests and I just go ahead and save it this rebuilds the container image it’s going to happen right away now this might take a few moments because that’s exactly what does happen when it comes to building out containers but there it is it rebuilt it and then it restarted the application altogether so the next thing is actually changing the code but before we do that let’s take a look in here we see Hello World at Port 802 now inside of main.py if I change this to hello world Earth or something like that we should be able to refresh in here and it automatically changes which is super nice so really the command we want to use from now on is d-at in here so that we can actually just change our code we now have a Docker based development environment all you’ll need to run now going forward is Docker compose up and watch and then if you do need to get into a command line of this Docker image you can do so with simply Docker compose r and then the service name which is in the composed damel in our case it’s simply app so if we come back into our read me here we can just go ahead and say app and then you can put in something like /bin/bash which will allow you to go directly into the command line shell now you could also see something like or uh you could do something just like that but instead of b/ bash you could just go ahead and say something like python right that should actually work as well which will give you the virtual environment version of python so let’s go ahead and try either one of these out so you can take a look at what we’re doing here and then we’ll be able to see that I have this runtime ability in here as well so every once in a while you’ll see this removed orphin you might want to just add that in to a command as well uh whenever you run it but we’re going to leave it like this here is that version if I import OS and then I print out os. get current workking directory I think that’s the command we hit enter and we see that we’re in code if I go to exit it it exit the container altogether so what I typically do is not run python directly but I want to go into the command line kind of like sshing into this container to then run off of that and then in here I will be able to list things out and there’s main. Pi which is quite literally this code right here and so we can actually see that as well by doing cat main.py this will show that code in there if I were to change main up high in here to something different to like Hello World and then let’s clear this out again and then run cat main.py it changes and shows that actual code right there as well so now we have a mostly complete development environment that we can use through Docker um there are other things that we might want to expand on this but if you’ve ever thought about hey I want to use a more even more isolated environment inside of my project this would be the way to go but the main thing about this is mostly to be prepared for production which I think we are well prepared for production now in the next section what we want to do is actually start using another service in Docker compose to actually start building out the data schema that we want to use the key takeaway from this section is that we now have a production ready development environment of course this is thanks to Docker containers and this Docker file right here now the composed file helps us with the development environment the docker file file will help us with the production environment but both of them together give us that production ready development environment now this is more General right it’s not necessarily about our python application which of course means that we also need to make sure that our python application is ready for a local development environment as well which is why we started it there some of you may or may not use the docker composed stuff during development and that’s totally okay the key thing is that it’s there it’s ready and we can test it when we’re ready in other words if you need to just run your uvicorn and activate your virtual environment and run that on your local machine like that that is totally okay and it’s more than acceptable when it comes to the actual Docker compos though we can also just have that running if we need to because the way the actual Docker composed file is developed is it will react to changes that you make with your code or at least sync those changes and a lot of that has to do with all of the different commands we put in here so like we saw when we changed our requirements.txt the entire Docker container was rebuilt and then started to run again thanks to Docker compose watch so this really gives us this Foundation that we can build off of now does this mean that we’re not going to change anything related to Docker or Docker compose no as our application develops we might need to change how it actually runs whether that’s in Docker Docker compos or whether that’s locally just through Python and virtual environments but the key thing here is this Foundation can be used on other python applications if you want and with slight modification to the docker file you can also use it in node.js applications now node.js applications will also follow this same sort of pattern that’s the cool thing about the docker file itself is that it has well very stepbystep instructions that are going on here and of course you could use this to deploy things manually as well so if you don’t want to use containers when you go into production this Docker file will at least help you with that as well so we’re really in a good place to start building on top of our application and really just flush out all of the features we want to use now we still will use Docker compose for other things we still will use the docker file for other things but the point here is we have the foundation ready and that was the goal of this section let’s take a look at how we can start building new features on our application to really make it the true analytics API in this section we’re going to be building out API routing and data validation we are creating something called a rest API which has very static endp points that is predetermined URL paths that you can use throughout your entire project and so fast API is really good at building these things which is the reason we’re using it but the idea here of course is you’re going to need to have something the code going and I’m going to be using my Docker compose version of this with watch running so if you don’t have Docker compose up and going you can use just straight python of course what you’d want to do is clone the repo itself and if you are going to use straight python you’ll create a virtual environment in there and then you’ll navigate into SRC in which case you will do the docker compose command which is this one right here and you would just maybe change the port or you could use the same Port so in my case I have both of these things running and they are allowed to access so either one works the point here is being able to develop on top of what we’ve already established previously and so we can actually build out the API service now there’s one other thing I’m going to be doing here is I’m going to be using something called Jupiter notebooks and I’m going to go ahead and create a folder called NBS here and it’s going to be simply hello-world doip python notebook or iynb and so in here I’m going to go ahead and create some code and print out just simply hello world and then I’m going to go ahead and run this with shift enter this will then prompt me to select my python environment which is going to be my local virtual environment right here this of course means that I’m not using Docker for this the reason I don’t want to use Docker for this has everything to do with how we’re going to test this stuff out and so I’m going to go ahead and install this this uh you know it’s asking me to install the iPod python kernel in here and we want to do that obviously you can add this to requirements.txt but cursor VSS code wind Surfer all really good at actually just running Jupiter notebooks right in line which is why we’re going to use it right here so this is pretty much it for the intro Now using these jupyter notebooks we will then test out all of the API routes we put in and make sure that they’re working as intended let’s jump in the purpose of rest API are really API Services is so that software can communicate with software you’re probably already well aware of this but since we’re having software automatically communicate with each other we want to make sure that we Implement a health check almost first and foremost so that if the software has a problem reaching it it could just go to the health check to see if the API is down or not so that’s where we’re going to start and this is a really just a lwh hanging fruit way of seeing how we can build all this stuff out now inside of Main Pi I’m going to go ahead and copy my read root here and I’m going to paste it underneath everything and I’m going to just change the path to Simply Health with a z health check API endpoints are usually just like that it’s not usually Health but it has a z in there not really sure why but basically what we want to do here is rename the function to something different and we’ll go ahead and say read API Health something along those lines and we’ll go ahead and say status being simply okay in other words we can tell the designers that are going to be using this API hey if you need to just go ahead and run this health check this is also going to be really important for us when we go into production so now what we want to do is actually test out this health check so inside of my NBS here I’m going to keep this hello world going and I’m going to import a package called requests so python requests is a really nice way to do API calls this might be what you end up using in the future there’s another one called htpx both of them are really good and have a very similar way of doing HTP requests now what I see here is I actually don’t have the module called requests or python requests itself so jupyter notebooks one of the other nice things about these sort of interactive environment is I can run pip install requests right here and what it’s going to do is it’s going to use the environment that you set up at the beginning of this one right it’s going to use that virtual environment to install the packages you might need in which case you can just run something like that every once in a while you might need to restart the kernel which you just hit restart and there it goes once it restarts you have to run the various things and at this point python request is installed on my local virtual environment okay so I’ll leave that out for now I don’t actually need it any longer so feel free to comment these things out and realize I’m doing shift enter a lot to actually run each cell itself okay so how do we actually call this health check itself well the way I think about it is my endpoint the endpoint I want to use which is going to be Health Z and then I’m going to go ahead and do like my API base URL which I often do something like this base URL and it’s going to be HTP col Local Host and then the port we’re going to be using in my case I’m using Port 802 now if you’re not aware Local Host is also the same thing as doing on 7. 0.0.1 those two are interchangeable with in most cases okay so endpoint maybe actually is not what I want to do I want to say path here and I’ll go ahead and say the endpoint is the combination of these two things which I’ll just use some string substitution to make that happen just like that so now we’ve got our endpoint here and I can print that out and that’s pretty straightforward now if I could do a command click or control click I can actually open up the web browser and see this right here now of course we want to build towards automation since we’re using our our uh you know our API service we’re going to go ahead and say the response equals to request.get this endpoint right here and then all I want to do is print out response. okay okay so I’m going to go ahead and run this and what it does is it gives me a True Value great if I change the path to something like ABC on it and run those two cells again I get false fantastic okay so I’m going to keep it in as just simply health and this is what we’re going to be doing we’re going to be building on top of this with different paths and different data points and also different HTTP methods now let’s go ahead and create a module for our actual API events so inside of our SRC here I’m going to create a folder called simply API inside of that folder I’m going to go ahead and create another one called events and then inside of there we’re going to go ahead and do routing. so what I also want to do is make sure that each one of these things has an init file in here to turn events into its own python module so we can use dot notation appropriately as we’ll see in just a moment the idea here is we want to have very specific routes for our events resource so if you think of main.py this is kind of generic routes what we really want to have is something like SL API events and then that being all of the events stuff that’s going on in here so very similar to main.py but just slightly different in the way it’s termed so let’s see what that looks like so we’re going to go ahead and do from Fast API we’re going to go ahead and import API router and then I’m going to go ahead and declare router equaling to the API router now this router right here here is basically the same thing as the app but it’s not an app it’s just for this one small portion of the larger app and so in here then I’m going to go ahead and do router and we’ll go ahead and do.get the HTTP method of get which we’ll see in just a moment and I’ll just put a slash here and then we’re going to Define and this is going to be something like you know get or let’s say read events eventually in here that we will have some sort of more robust response and I want to return back just let’s go ahead and return back in a dictionary of items and I’ll just go ahead and say one uh two and three now the data that’s coming back we’ll definitely look at and change over time but for now we’ll just say read events like that so in order for this to work what we need to do is we need to bring it into main.py because right now the actual fast API application doesn’t know about this routing module because we have nowhere it’s not orted anywhere right this right here has all of the definition that we have in place at this point so the way it works then is we need to import it from API so we’ll go ahead and do from API and then the fact that it is a python module because of this in it we can do do events like that and then I actually can also then go one more do notation and import routing and then we can import the router that comes in and then we can change it as something like as event router now the reason that I do as event router is so that I can just riable name this if you will um so that I can have a lot of different ones with the same sort of organization or the same sort of setup that we might have had before now that we’ve got that we can come in to our app itself and we can use do include router and we can just pass in that event router but as you recall the event router itself has a slash here but we really want this to be SL API SL events like that so this is the prefix we want to use for this route now of course the way I can actually use this path is I could set it right here and that would generally be okay it would technically work but what I actually want to do is go back into main.py and set the prefix itself to that actual prefix without the trailing slash okay great so that is now an API endpoint route that we can use of course we still need to test these things out and we will but before I go much further I will say that this routing is something I actually want to change the way we change that is by jumping into init.py and in here I can do from routing import router and then I can just go ahead and exported by putting it into this all list and we just put in router just like that notice it then gets highlighted and now I can just come back into Main and I can get rid of routing and just imported that way it’s just a subtle way to do that it’s a nice thing that those init methods are able to do now the init method in here I could probably also import that router in this init method as well I don’t want to do that I just want to keep it like somewhat isolated from each other um or somewhat packaged in this way but now that we’ve got this router what we can do of course is jump into our API endpoint and just check it out by going API and events and there we go items 1 2 3 since we have this API Ino let’s go ahead and create a notebook to verify it so I’m going to go ahead and do one- verify router or let’s say API Event Route something like that IPython notebook will bring this in here now I’m going to go ahead and continue to number the notebooks themselves so you can always reference them later but the idea here is we want to do something very similar to this hello world where we’re going to go ahead and import the requests here so let’s go ahead and open up new code then we’re going to go ahead and bring in all of these inpoint stuff in here so we can do some various tests itself now I want to run this with shift enter it’s going to tell me what environment I want to select again we’re going to use our local virtual environment and then our endpoint our actual path is going to be a little bit different than this and that of course is going to be API events and we can have a trailing slash in here and then I can do my response equals to requests.get and then that path itself now the.get method here is correlated directly to this git method right here we’ll talk about that more in a little bit but for now I’m going to go ahead and run these two things and what we should get is the actual API endpoint but of course I did path not endpoint so that’s a little slight little mistake so we need to change it and then that gives us back a okay response which we can now print out with response. okay and there we go we got a True Value there so what we can do then is say something like if response. okay like that we can actually print out the data that’s coming from that response by doing response. Json like that and then we can print out that data okay so we hit enter and there we go we got items 1 2 3 now the reason we’re doing response. Json is because this is now a dictionary so if I actually do type of that data we should see that it is a dictionary itself right oh this is not dot but rather comma we can see that the class is a dictionary which in other words means that I should be able to do something like data. git items and do the dictionary value stuff that you might want to do now of course if we change our API endpoint this might have some issues in here as in it items might be none so if we actually did change our API end point let’s change it real quick to being you know uh results something like that slight change made it happen real fast I refresh in here this is still the same data because I didn’t actually do the res the request again so I didn’t actually hit the API endpoint again now when I do hit it it gives me something a little bit different in terms of the actual data that’s coming back so this is one of the main reasons why it’s really important to think through all of those API end points in the beginning you want to make sure that these stay basically the same going forward that’s why we’re going to be testing them out first to make sure we’ve got exactly what we want then we can always of course improve it later but something is simple as just returning back results here could have a drastic effect on somebody who might want to use your API itself even if that somebody is just you let’s keep going we want to take a look at the impact of data types on our rest API now what we saw in the last part was when I changed the key value of this data from items to results the actual notebook itself had a different result altogether and so I actually copied that notebook and just made it into one or two cells here what we run now is we can see that result still so what I want to do though is I want to take a look at how this impacts a single API result itself so in other words I’m going to go ahead and copy this and we’re going to call this instead of read events we’ll call this get event like a single event itself and then we’re going to return back what that single entry might be so if you think of this in terms of a single row and this being a you know list or a bunch of items in a table more on that soon but the idea is we want to look at this single row here now the way this works is we put in something like event ID so this is going to be a you know a variable that’s going to be passed after that endpoint here that variable will then come into the function itself so whatever you name that you put it in here and of course if you declare it as a data type like int then it’s going to look for a specific number we’ll see that in a second and then of course we can return back the ID being that event ID okay so back into this new notebook here I’m going to go ahead and pass in let’s say 12 in here for example and I’m going to go ahead and run this and what I should get back is exactly this data it’s still a dictionary but it’s now returning back ID of 12 if I change it to something like a and then try to run this I get nothing back in other words I can print out the like you know okay and then do response okay and I see that it’s not okay and I can also open up the URL itself and we see that it says un able to parse string as an integer because the input was a the URL everything like that we’re seeing that the input is a so there’s a couple ways on how we can solve this number one we could change this back to being a actual number because it’s an invalid request number two if we did want to support an a we would change this from a integer right here a very specific data type to string which is a little bit more generic in the sense that we can then you know now support that API inpoint for that string now in my case I want to keep it back as an integer because that’s how we’re going to approach this inpoint itself which it requires it to be an integer all across the board which means then our notebook will also require that as well but we’re not quite done yet that’s the input that’s the data coming through what about the data going back like what if I change this to being ABC ID and run it that way what we end up seeing here is we will actually still see the data come back and in some cases oops let’s make sure we save that and then run it again and there it is now this is now the output data it is now incorrect so we need to change that to something different that something different is going to be called a schema which is basically just designing the way the data should flow in this case we’re going to be using pantic so we’ll go ahead and create schemas dopy and I’m going to go ahead and do from pantic we’re going to import the base model and this base model is going to help us a lot so if we look back into our requirements.txt we did add pantic in here pantic itself most likely will be in with fast API as well so we’re really just using the basics of pantic here and we’re going to go ahead and say the event itself is going to take in that base model and this is where we’re going to design how we want it to be returned in this case we’re going to do an ID which has an integer so that’s the schema that is going to be used this is also a lot like data classes but it’s going to end up turning into something along these lines where it’s a dictionary that’s coming back so that’s kind of what’s expected by this schema now the way this ends up working then is back into our API routes we can import that schema so from schemas import the event this is probably better than calling it event we would call it something more like event schema so let’s call it that you could call it model but we’re not going to and you’ll see why when we start using the database stuff database models is how I typically call models schemas like the design that might go into a database model or come from a database I’ll call schemas okay nevertheless we now have this event schema in here and it’s in here as well now what we can do is we can actually just return that data type so if the colon and then the int is how we declare the incoming data type this arrow and then the data type is how we declare the outgoing data type the response data type itself and so as soon as I do this what I should probably see inside of my fast API application or at least very soon I should see some error with that data type okay so let’s go ahead and do a request here and I’m going to go ahead and run it in two places so I’ve got my notebook here and then my fast API application running below as soon as I run that I see that it’s missing the response ID field required the input again is ABC ID of 12 so the input’s incorrect so all we need to do then is change our route itself and that is going to be of course ID or basically to match that schema from before and now we can run everything again and this time it actually ends up working great so now we’ve actually made our system a little bit more robust it’s a little bit more hardened because it’s harder to make a mistake now or that mistake will be a glaring problem here so if I put in ID that is possible you make that mistake on accident but as soon as you go to try it out you’ll get that same error like what is ID don’t know got to fix that and we’ll go back okay great so then we bring it back just like that Co cool so of course solving errors also comes back to using something like git so tracking the changes over time will become important if you were using something like this if you’re building out a rest API of course you’re going to want to use get I’m actually not covering that because it’s outside the scope of this series but it is something important to note right now because git would help identify that problem because then you could just do something like get diff and of any sort of thing if it was already in your database so let’s take a look at that right now or rather in your data itself so I’m going to go ahe and add it and we’ll go ahead and do something along the lines of the actual name itself which I believe I called it this will be something like uh five and we’ll do basic data types there we go and so now I’m going to go ahead and do that slight little change here save it and now if I do get status I can see that there was a change and now I can take a look at that difference and what I should be able to see uh oops not schemas but rather the routing itself so get diff of that I now see exactly the problem that’s coming through so this would be another way to catch it and hopefully catch it early on of course you can also use automated testing using something like Pi test to actually catch that as well which would be even better to harden the system but the point here is we now have a way to validate a single piece of data how about a list of data let’s take a look now that we can return reliable data for a single item let’s go ahead and look at how we can do it with a bunch of items so there’s a couple different ways and how we could think about doing this one of the ways would be to do something like this where we just bring back a list you can just use the list element just like that and then put brackets there and put in that event schema that is a way to return back a list but that actually changes the result schema from a dictionary to a list so you can’t exactly do a dictionary like this that’s not really how it works so what we would end up doing is we would actually bring in a whole new schema for the result results themselves so what I would do then in here is inside of schemas dopy I would do something like maybe the event list schema in which case this is now going to be results of the data type itself which will be a list of schema right and we could also bring in the other class from typing so we can do from typing we can import the list class and we can use that instead of the list element so this is a better data type decoration itself instead of doing the python built-in list both work but this one is more verbose okay so now that we’ve got this we’re going to go ahead and bring it back into our routing and we’ll go ahead and use that list event schema here and we’re going to go ahead and return back that a list event schema and just like that we don’t need to change it at all because inside of this schema we’ve got this results here great so the actual result itself well let’s actually take a look at that one as well so if we save this I’m going to go ahead and duplicate this basic one and now we’ll do list data types or something like that we’ll go ahead and come in and change this to Simply events and we will update this and let’s go ahead and run it now and we get false once again okay so if I open this up I get an internal server error and what’s likely happening is the actual input is incorrect so each one of those items has invalid input data itself and that’s because the actual data that’s coming through in here is not an event schema so we would actually want it to be at least of that type of event schema in other words it needs to look closer to this for each instance in here the way we do that then is actually turning these into dictionaries themselves so ID equals to that ID and then we would just repeat this process for each element which I’m just going to go ahead and do really quickly and of course I probably could have just copied pasted but there we go so now the results are dictionary values and we can see if that solves the problem for us it looks like the application rebooted and now we can go back in here and we can run it again now the results are coming back so this is actually super nice so that means that our routes themselves are now hardened to what we want for our results and of course if we wanted to add something to this list scheme I say something like count and we’ll go ahead and do count being something like three three right then we would go back into our event list schema come in here and say count and this is going to be an integer H here uh we could also have it as an optional value but we’ll just leave it in there just like that and then we’ll come in and run this again and now we’re getting false once again so let’s see for instance uh we don’t have the results coming back correctly so let’s go back in the route we’ve got our count in here uh this actually should work just fine let’s make sure that everything’s all saved up and let’s try that one once again with it all saved and there we go so now we’ve got the data coming back here it is with that count in there so the point of this of course is to make sure that how we’re designing our API when it comes to communicating the data is going to be very hardened it’s not going to change a whole lot altogether so of course this is going to be even better when we start entering this data into the database uh and also ex and actually grabbing the data from the database but before we even start grabbing the data from the database we need to learn how to send data Beyond something like the URL route in other words we need to take a look at the postp put and Patch methods to see how we can send data to our API let’s take a look up until this point we’ve been using the HTTP git method this becomes even more obvious when you look at something like requests.get and then inside of main.py we’ve got app.get and then we have a URL path and then in routing. we have router. git so the G method is very very common so we’ve got git like this we’ve got get down here for like some ID those methods are what you’ll use to grab data from the database or from your API itself and so our notebook is really just a small example of that this notebook is really meant to be like as if you were building out another app to work with the API we’re also building at the same time but now what we need to do is we need to be able to send data back to Any Given uh you know server or to our API so what we want to do is we want to use the post method that’s what we’re going to now is we’re going to use that post method in here and so the idea then is we’ve got our endpoint all of that stuff can be the same to change the response here all we need to do is change this to not get but rather post so now this is going to attempt to send data but in this case it’s actually not sending any sort of data at all because we well we didn’t Define any data to send so if we actually look at the response itself so I’ll go ahead and do response. text and we can see detail method is not allowed as in the HTTP method is not allowed if we do get and we see that as well we can see that it actually unpacks two different things so we’ll look at this as well in just a moment but the idea here is going back into the post method we get a detail not allowed that not allowed error is because of our routing here so we don’t actually have a route to handle the HTP method the only routes we have right now are to handle git Methods at these particular endpoints if we want to handle a post method as well we can just duplicate our function here and then just change it from router. git to router. poost which is now going to be this as an send data here type of thing and then this instead of read event it will be something like create event now the important part to note is the endpoint is the same as each other but the method is different so inside of fast API we create a different function to handle the different HTTP method as well as the different HTTP route sometimes what you’ll see on AP apis is something more like this where it’s create and it’s quite literally a different route as well to handle the different method now that is not nearly as common in the case of creating data so we’re going to go ahead and leave it in as this right here so we’re going to get data this is going to be more of a list view this is going to be more like a send data here or a create view right and so let’s go ahead and get data here type of thing right so the idea then is in this function when we go to create an event the response should be very similar to the detail view because when we’re going to add data into the database what we could could return back is hey we added that data oh yeah and here is the data we added because if we modify things inside of this function which we might then we want to send back whatever that modification would end up being so in my case I’m going to return back the original event schema and we’ll just go ahead and say we’ll just do an arbitrary ID here because we don’t actually have a database that will help us with that ID just yet okay so we’re almost there now that I’ve got the actual inpoint let’s go ahead and try it again because now the method should be allowed so let’s go ahead and run it again and now I get the method being allowed and then the response is just echoed back in here so the post method is basically the same as G method at this point it’s just echoing back some data but that’s not what we want to do we want to actually send data here we want to actually see some data so I’m going to go ahead and say data and I’m going to put this equal to an empty dictionary in here and I want to actually print out that EMP empty dictionary or rather data dict equals to empty dictionary rather so I’m basically declaring the data type and then an empty dictionary in here so we want to see what that data is and so this is actually very similar to what we saw down here where we passed in an argument into the route a wild card if you will into the route it’s very similar to that but slightly different okay so let’s go ahead and take a look at that and we’ll do this by sending that event again and if we look in here we see that there is no data coming through so there’s that dictionary okay so if I actually want to send data though I need to change things a little bit before I do though I’m going to go ahead and jump into the response and grab the headers that are coming in here and so what we see is a bunch of headers but the important part is the content type here so what’s happening on the server side in our fast API application is is expecting a certain data type to be sent especially with sending out data in other words the data itself is going to be a Json data type so I’m going to go ahead and say something like our page is equal to you know Test Plus or something like that so that’s the data we’re going to send now and we’re going to go ahead and say data equals to data but there’s a caveat here so if I actually run this now I will get an error right so it says the error there and of course I can say something in here something like else print out the response. text so we can see what’s going on with that response and what it shows me is input should be a valid dictionary and here is the actual input that was sent so that is not Json data we can know what Json data is in a couple of different ways so one of the ways we do this is by bringing in the Json package that’s built in to python itself and then we can do json. dumps and put in data there hit enter and now we see this string in here this is actually quite literally a string which you can check by just putting type around it and you can see that as a string it’s no longer a dictionary like what we have up here so so in other words this is string data but it’s in a very specific format that format is you guessed it Json data and so we can send this data now to our back end so let’s go ahead and try that so I’m going to go ahead and use instead of just pure data here I’m going to go ahead and grab this Json dumps pass that in run it now this time I didn’t get that error it actually did send that data end and we can actually take a look at that data with this right here so what ended up happening is this data was treated as Json still even though I didn’t tell it to treat it as Json so one of the other things that you do every once in a while is you create headers and you declare what the data is that’s coming through in this case it’s application Json this is not the only kind of data so for example if you were sending a file like an image file you would not use application Json you’d use something different but the idea here is you can pass in headers in there as well and send that same data these headers are going to send to our back end our actual API will look for those headers and be like okay cool I see you headers you say application Json so I can treat this data as if it were Json but in in the case of fast API it actually unpacks that Json data for us and turns this into an actual an actual dictionary which we can verify by coming in here and saying type of what that data is and then we can come through in here send that and there we go we’ve got a dictionary in here so in other words what’s happening here under the hood this is all happening for us but we still need to understand what’s going on when it comes for apis communicating with each other it’s usually through Json data it’s not always there’s other data types that you can pass but it’s usually through Json so fast API was designed in a way that said hey when we skit post data we’re just going to expect it to be Json so then we can just infer that the data that’s coming in is going to be a dictionary in other words I should be able to respond with this data assuming I did it correctly but of course the schema that I have set up won’t allow for this data to respond as an echo so I’m just going to get rid of this schema for just a moment so we can see the Echo come through now so with data test I’m going to go ahead and run this all again and now we should see that Echo coming back as soon as I bring that schema back it is going to run an error as we’ll see in just a second so if we run that back everything’s saved run it again what I get is now an error because the schema is invalid so we need one more step with our data and that is going to be an incoming schema in other words this can’t be generic like this it needs to be something else so for now I’ll leave it like that and we’ll look at that incoming schema in just a moment now I will say one of the cool things about this as well is we can go even further I can copy this router here down here and we can grab something like put as in we are going to go ahead and update this data now I’m not actually going to cover patch but put is very similar to this where we have the event ID coming through and then we also have the actual data that would be coming through from the back end this actually would be not called Data but rather payload that’s what we would end up calling it for the put events so we’ll see this one as well but in this case we are just returning back this here and this would be something more like update event now in the case of an Analytics tool I don’t know why you would be updating the event to directly instead of just entering a new event but this is kind of what we need to understand is these different methods now there are other methods as well that I’m just not probably going to implement for a bit but one of them being like delete where you would delete a specific event itself that would be another one that you might end up using once again we’re not going to be implementing that just yet but now we want to actually validate the incoming data to our API we need to ensure the data that’s being sent to our API is correct or at least in the correct format so very similar to when we send data back needs to be in the correct format in this case as the event schema we need to make sure that the data that’s coming in is in the correct format not just in the correct data type so not just a dictionary for example so what we want to do here is create a new schema that will help us with this validation now we already saw some elements of this validation with the git lookup where we said event ID is an integer we want to make sure that it’s only a number of some kind you could obviously change it to being an Str Str in which case that actually sort of breaks this endpoint where it’s now a wild card it’s no longer an actual number so we want to keep that one as an integer the same sort of concept exists for our create event as well so what we want to do then is we want to jump into our schemas here and I’m going to go ahead and create a brand new schema this one is going to be called our event create schema this one will no longer have an ID in there but rather it’s going to have a path in here which for now will just leave as a string value and so this schema itself is now going to be brought into the router in here so we bring it in just like this I’ll go ahead and pass it in here and we’ll do something along these lines right here there we go and so now this is going to be our schema so this is the data in here all I can do is bring it in as create event schema or the event create schema and then I would want to actually rename this to Simply payload you could keep it as da data you could keep it as a whole lot of things but I actually want to keep it in as payload I find using the term payload to be a little bit better than data data is a little bit too generic for what this kind of data is payload is what’s being sent to us right it could be correct data but realistically it’s payload just like what we started talking about down here now the other thing is with the put method we could could do the same schema if we wanted to or we could expand it if we wanted to add something different so let’s say for instance we came in here and did the event update schema and instead of updating the path maybe we just update the description and that’s literally the only field that we allow to be updated in an event update schema and so these are required and the only fields that are changing so what that means then is I’ve got three Fields here that the event could potentially save that is going to be these three Fields right here right so we would have ID we would have path and we would have description so this is what would actually be stored on the database where the actual endpoints are only supporting these items in here but the actual event schema the final one might have all of them as well so we’ll talk about that when we get to the database stuff uh but now that I’ve got both of these I’m going to go ahead and bring that one in as well and we’ll bring it in to the update portion also so let’s go go ahead and do that just like that great and this time I’ll just go ahead and print out that payload also so I want to see these two events and see what they look like going back into send data to the API this notebook here I’m going to go ahead and run this and what we should get back is we’ve got an issue here for our input right so we’ve got page in here and the input saying Page Field required being path not page so that means that I need to change this to being path or I need to change the other other schema being page so I’m actually going to change the schema because maybe I don’t want it to be called path maybe I want it to actually be called page and so in which case I would save it like that and then we go back into the API call in the notebook and now it actually sends it back to us which is great so of course we could always Echo the data back as well if we wanted to by sending or updating the event schema payload altogether we’ll look at that in just a second but what I actually also want to see is this update view on how that might work so going back and here we’ll go ahead and copy some of this data here paste it below this time I’m going to go ahead and grab the endpoint stuff and it’s going to change slightly uh which will be this down here and then we’ll say this is 12 now with a trailing slash because of how our endpoint is set up and so there we go and I’ll call this my detail inpoint so detail endpoint and then detail path and we probably don’t need this base URL anymore and then we’ll go ahead and put this down here now you might be wondering what method do we use hopefully you remember it’s simply the put method that’s it okay so now that we’ve got this there’s one other thing that I can do in here that’s actually really nice so instead of doing data equals to Json dumps data and then the headers I can literally just use Json itself python requests will then Implement all of those other things for me and what we said was description is the value that we need here in which case I’ll just say hello world and then now this of course is an HTTP put I now can run this and I should be able to see that it is okay and if I look in the terminal of our application running I see there’s the description Hello World and there’s the page and of course back into the route this is not a dictionary now but it’s rather a schema so if I wanted to get this stuff I could go ahead and do payload page and then in here it would be payload uh what did we call it maybe that one was page yeah so that one’s page the other one’s description so this is payload do page this is going to be payload do description giving us the full advantage of using pantic in our API and so back into the notebook itself if I run that again I see that it says hello world for that description and then the other one should probably say something like uh the path itself so let’s go ahead and run that again and there it is cool so we now have the routing and the schemas for incoming validation as well as outgoing validation to ensure all of that stuff’s working with pantic we can have optional fields at this point we don’t have any optional Fields we’ve got only required Fields so let’s see how we can actually create these optional Fields now before I do I want to actually update my notebook here a little bit so that the create endpoint and the update endpoint are basically one sell by themselves so that there’s no issues with it and so the create one is going to be here I’m going to just change this to create endpoint and we’ll go ahead and put that there the data will actually turn this back into Json data and we’ll pass in the dictionary with the page of you know slash Test Plus or whatever uh this is mostly so we don’t have any conflicts with some of the variables that might come through from the different responses and then also we only have to run uh each one of these one time right so get update great so now that we’ve got this let’s go ahead and add in the optional data here so jumping into our schema what’s the optional data I want is in this event schema I probably want to have page and description optional I’ll start with just simply page and so the way this works is if we put it in as page colon string that is required data if we want to make it optional we bring in the optional class from typing and then we actually bring in the square brackets like this now it’s optional or at least seemingly optional so let’s go ahead and save everything jump back into our notebook and then run it I get an internal server error so if we actually take a look at that server error what we’ll end up seeing is the field is required right so it’s now still assuming that that field is required so it’s not quite optional yet uh so let’s go ahead and add in a default value for it so it’s almost optional just not quite there my default is going to be just an empty string here just like that so let’s make sure it’s saved and once again back into our notebook we’ll go ahead and run that post and this time it seems to have worked and there we go okay so what I want to do is actually Echo back that response I want to see that data I don’t have an error anymore with that field but it doesn’t seem to be showing up much right so if we look in here we’ve got the response coming back let’s actually take a look the response coming back is Page being an empty dictionary so that’s actually kind of nice in the sense that yeah cool the page is there if we look at the other one same thing the page is there it’s empty great so what I want to do then is I actually want to return back the page data at least from the first one and the way we do that is by jumping into routing and then putting in the page data in here and we can do that with the page value being added to the response value so payload dopage we save everything like that we go back into our notebook we run it again and now we can see that page values coming through and now it’s technically an optional value for this field right for that data and we can do that same thing again with with the description itself and we can come in here just like that and now we’ve got some optional values in here and if I look at my requests there we go and we can come in and we see all that great so of course I want to Echo back the description back here as well so back into our rounding here we’ve got our payload down here which has our description and then I can do something along these lines where it’s pay. description great so this will help us with that echoing and of course we can verify that again and there’s that description coming through of course it’s just echoing what’s ever here and that’s that okay so in the long run of course these schemas would be tied into what’s happening in the database so in the middle of all this stuff would be sent to the database and then receiving stuff from the database but the other thing about this is the actual payload itself we can actually return it to be a little bit easier on ourselves so now what we can do is do data being payload do model dump and what this does is it takes the payload and turns it into a dictionary which is basically from pantic itself has that model dump right and it used to be something different I think it was like two dick maybe uh like this if I remember correctly but now it’s just simply model dump in which case this is now going to be a dictionary in here which I can unpack just like that and I would do it also down here as well so we would unpack it like that and then the data just like that great probably don’t need those print statements anymore uh but now we’ve got the data coming through as we probably want okay so once again I can verify these in my notebook and I should be able to see the echo data coming through in here and one of the questions of course would be oh well could I actually send the description now if I wanted to so if I come in here and do description and set something like you know abc123 will that respond that data back and the answer right now is no hopefully you understand why at this point but I’ll let’s go ahead and go through it first off the data that’s coming through is just this payload right here it’s going to ignore that other data if you pass in more data in here it’ll just ignore it there’s probably a way around that as in it will then say Hey you have too much data there’s probably a way to do that but for us we really just want to make sure that we’re grabbing the correct data and so I can actually come up here and give that same optional description in here and also on the update view if I wanted to be able update the page I can do that as optional as well but there we go so now that we’ve got the description coming through I should be able to Echo that back as well and there it is it’s now being echoed and now I have a way to change the responses really quickly and it all has to do with these optional values if you wanted to add in optional values now every once in a while you might want to add in a default value this default would be then using a field so you’d come in here and say something like field default being an empty string or my description something like that in which case you would come back into the API let’s go ahead and get rid of the description I passed in and now we’ll take a look at it if I run it again it now gives me that default value if you wanted to put one in there um and of course there’s more verbose ways and more robust ways we can continue to modify that stuff uh but overall this is actually really nice because I have the ability to change all sorts of things in here for for my schemas and now once again adding to that stability to how we end up using pantic inside of fast API our API now has the ability to handle different HTTP methods like getting post at different API endpoints like our list view or our detail view now once we actually can do that we also see that we can validate incoming and outgoing data now a big part of the reason I showed you schemas in this way like with multiple Parts is because the way you use different data pieces might change depending on your needs for your API service when it comes to an analytics API do we really want to ever update it maybe maybe not this is going to be something you’ll decide as you build this thing out more and more the point here though is we have a foundation to move to the next level that next level is going to be using SQL databases to actually store the data that’s coming through and be able to handle it in a very similar way that we’ve been doing and the way we’re going to do that is by using SQL model if you look at SQL models documentation and you actually take a look at how it defines a database table it looks like this this hopefully looks very familiar to what we just did and that’s because SQL model itself is based off of pantic it’s also powered by pantic and something called SQL Alchemy SQL model is definitely one of the latest Cutting Edge ways to work with python and SQL SQL of of course is designed for storing data it designed much better than python would be python itself is not a database SQL is and so we’re going to use SQL model to actually integrate with a postgres database called time scale DB and another few packages to make sure that all works really really well let’s take a look at how to do that in the next section in this section we’re going to be implementing SQL model this really just means that we’re going to be taking that piden schema validation for validating that incoming or outgoing data and we’re going to convert that so that it actually stores into a SQL database SQL model will do all of the crud operations that’s create retrieve update or delete of that data into that SQL database now the SQL database we’re going to be using is postgres postgres is one of the most popular databases in the world for good reason one of the reasons is the fact that it has a lot of third party extensions to really pick up where postres Falls flat one of the things that postes doesn’t do very well is realtime analytics or just a lot of data ingestion postris doesn’t do this natively by itself in which case time scale picks up the slack in a big way so we’re going to be using time scale because we are building a analytics API and this is designed for real-time analytics and it is still postgres so at the underlying technology how we’re using SQL model is going to be the same regardless of which version

    of postres you use especially in this section where it’s really just about using SQL model itself we will use the advanced features that time scale gives in the next section this section is really just about understanding how to use SQL model and actually storing data into a postc database instead of just using pantic to validate data because that doesn’t really provide that much value let’s go ahead and dive in on our local we are going to be using Docker compose to spin up our postgres database we’ll start with the configuration for Pure Old postgres then we’ll upgrade it to time scale and all of this is going to be done using Docker compose in just a moment now if you skipped Docker compose you can always log in to timescale decom and get a database URL and just use the production ready version right in there that process we will go through in the next section but for now I want to just do the post one on my local machine now a big part of this is looking for the postres user and all of the different environment variables that are available so we’ve got postgres password user and DB those are the main ones we want to look at to get our local version working inside of Docker compose so I’m going to open up compose diamel here and then we’re going to go ahead and tab all the way back make sure it’s on the same level as app one of the ways you can figure this out is by just you know you know breaking it down like that and then you can do something like DB we’ll call it DB service in here and then I can go ahead and create those things out with this still broken down having it broken down makes it a little bit easier to know exactly what to do now the first thing is we can come in and decide our image so if I were to use postgres itself I would come into postgres official and then I would want to look for the tag that I’m going to end up using in here and be very deliberate about the tag especially with post cres as a database so maybe you use 17.4 Bookworm or just 17.4 the size of them looks about the same so it really probably doesn’t matter that much between the two of those so you could do something like this postes 17.4 next up you would want to set the environment which will be all of those environment variables so once again searching for that postcg user in the documentation on the read me let’s go backup page here and search for that and you can actually see the different environment variables so you’d come in here and do something like postc user and then the next tab would be postres password and then something like this and then the actual database itself the core values here are basically this right so those are the key ones that we’re going to want to have next up what we’re going to want to do and this is one we will definitely need is something called a volume this volume is going to be managed by Docker compos we don’t manage it oursel it’s not quite like what we did with our app where we actually mounted the folder here into the Container instead we’re going to let Docker or actual Docker compose manage the volume itself so before I even declare that I’ll go ahead and do volumes and this one I’m going to call this my time scale DB data and just like that that’s all you need to put in and then you can use whatever you name this and you can bring it up to your volumes here and then have it go to a specific location which in the case of the postgres location the postes database it’s going to be VAR lib postgres SQL and then data and so that allows your postgres instance on Docker compose to persist data because you definitely want to have that so if you take down this or run Docker compose down all of this will still be up and running so similar to like our app itself we also want to declare the ports in here and I’m going to go ahead and use the default ports which is 5432 and then 5432 and then in some cases especially with what we’re going to be doing you might want to expose the database which is going to be 5432 I’ll explain that when we start to integrate this into our local project here okay so the idea with this is this is using postes of course right so if we actually scroll back into postgres there’s the name of it all that it’s all good we could totally go that route but I actually don’t want to use postes I want to use time scale so time scale is not a whole lot different but the thing is we grab this right here so now it’s going to be time scale that’s going to be the repo name or the image name and then we grab the actual image tag which we come in two tags here and we can actually scroll down actually in the overview here we also see the tags the notable ones latest pg17 so in the case of postgres itself you got 17.4 with time scale we can just do latest pg1 17 and everything else is the same which is really nice but of course I’m going to change this these values in here to something just a little bit different and I’m change the user to time- user and then we’ll go ahead and say time PD uh PW and then I’ll go ahead and do my time scale DB in here as the actual database name that I’ll end up using so all of those values are going to allow me to do something like this in myv which is going to be my postgres SQL plus pyop G which we’ll go over again once we implement it but it’s going to be our username so it’s going to be just like that and then colon our password like that and then at some host value which will come back to then the port value which is going to be right here and then slash the actual database itself so all of this is what we’ll need in our environment variables for our app itself so the app we can add it into up here we could say something along the lines of database URL equaling to basically that data now in development this is generally okay in production you’re not going to want to have your actual environment variables exposing something like this and so in which case you would use an EnV file as we discussed before but that’s when you would come in here and go into EnV and save it kind of like that now as we saw before as well when we were doing it with our app whatever you hardcode in here that’s going to take precedence over the EnV files themselves which is kind of nice but there’s our database URL and of course it’s based off of all of this stuff but there is one key component that we still need to change in here and that is this host value here this host value is going to be the name of the service itself when you are using it inside of Docker compose I’m going to show you how to use it in both places but I want to leave it like this for now we’ll save this and then inside of my Docker compose all I have right now is my app running so I am going to need to take this down and we’re going to go ahead and restart our app itself looks like I’m having a few issues with our app running so I’m not sure if maybe it’s related to this Docker compose rebuild it seems that it is now with the app down I’m going to go ahead and run Docker compose up and I really just want to see that our postgrad is a database is up and working our DB service is being created and downloaded and all that no surprise there it’s actually grabbing it from time scale the docker Hub version of the time scale DB of course it’s open source so that’s really nice and I’ll let that finish running okay so it finished running and it looks like everything is working in here we might need to test this out with postgres directly but in our case we’re just going to leave it as is and we’ll test it directly with python but if you are familiar with postgres you will want to probably test this the other thing I just want to make sure that my app is still running it looks like it is so I think our Docker compost stuff is probably good and we’re now ready to start the integration process within our app itself self I’m going to be integrating it in two different ways one is by using the docker compos version which is basically this right here another one is just considering our database as Docker compos so that our app itself will be able to run with things as well so in other words having basically two environments that can still work with this Docker compos instance of our database itself now one of the things that’s also important to note that we will be doing from time to time is we’ll go into the root of our project and we will run Docker compos down- v-v will delete the volume as well which means that it will delete all of the database stuff as well too the reason we run this especially in development is to get things right to get it working correctly then once it is done deleting we can bring it back up and everything is back basically from scratch when it comes to development this is a key part of it is making sure you can bring things down and bring bring it back up and it happens that quickly which is another reason to really like using something like Docker compose of course one of the bad things about it is you might delete a bunch of test data that you didn’t intend to delete which of course is something you want to think about if you are going to run Docker compos down one of the first steps in integrating our database is to be able to load in environment variables now when it comes to Docker compose directly the environment variables are going to be injected in other words we can come into SRC here and let’s go into one of our apis into one of our routes I should be able to go in here and go import OS and then just come on read events and I’ll just go ahead and print out os. Environ dogit of the database URL itself and so in which case I should be able to go into uh you know slash API SL events hit enter and I should see that print statement come through and there it is right there okay great so that’s being injected into our application because of this right here but unfortunately or fortunately this won’t necessarily work if we bring in our application directly from our virtual environment so let’s go ahead and do Source VV bin activate and then I’ll go ahead and navigate into the SRC here and then I’ll go ahead and run this same thing right here but this time I’m going to go ahead and not specify the port because it’ll give me most likely Port 8,000 so so now I can open that one up and I can go to API and events and hit enter the print statement is going to be none so we need a way to kind of handle both scenarios some of you might want to use Docker compos when you’re developing I know I do a lot of times but I also know that I also don’t do it a lot of times sometimes I don’t set up my apps this way so our actual fast API application needs to be able to be ready for both of those things so inside of requirements.txt this this is where we need to change something I’m going to add in something called python decouple now there are other ways to load in environment variables one I think is called python. EnV but I like using python to couple for a reason as you’ll see in just a moment so the idea here is now inside of my API I’m going to go ahead and create another folder in here we’re going to call this DB inside of DB here I’m going to go ahead and create that init method and then I’m going to also going to create config.py and what I want to do of course is using pip install python decouple I’m going to go ahead and do from decouple import config and then we’ll call this as decouple config mostly because this module I just created is also called config we don’t want to have any weird Imports in here now as you notice cursor is saying hey decouple is not installed so we need to make sure that we do have that installed in our virtual environment which I’ll do with that pip install there I already added it into requirements.txt so I don’t necessarily need to install it again through that but with this running now um I can reload in here and now I’ve got this decouple config in here so what I like to do is I like to put in our database and URL and this is going to be equal to decouple config and that’s going to be that same database URL and the default is going to just be equal to an empty string here so this is why I like python decouples we’ve got this empty string here the other thing about decouple config is it can actually load in a EnV file as in this right here so we’ve got this host value here so now hopefully if we did it correctly we should be able to use this now the way we use it is going to be from our events we now instead of using OS we’re going to go ahead and do from. db. config we’re going to go ahead and import the database URL or better yet I think we should be able to do from api. DB config Maybe grabbing database URL that way but we’ll see if that ends up working so let’s go ahead and take a look with that print statement now and going back in here we refresh and now we’ve got none from os. Environ but database URL is actually working inside of here which is giving me a different host value but that host value is based off of the EnV file so this actually brings me back to what I would actually do with Docker compose I would not put it hardcoded in here but instead what I would do is I bring in another EnV file like env. compose and use those same parameters in there and I’m going to just go ahead and put them in directly like that and then we would use maybe that same port value um it’s completely up to us in that term but we’ve got this database URL here and then in compose yaml the EnV file itself would be do compose and then I would just go ahead and completely get rid of this environment itself the only reason I’m commenting it out is so that well we can actually bring it in as need be okay so what we see here is we got this you know no module found error I think maybe I didn’t save something or whatever but let’s go ahead and bring it back up which should actually build everything uh but it’s not actually building the python decouple so what I’m going to do then is I’m going to go ahead and stop it um and then whenn in doubt when something like this happens you do uh d d build and that should actually build the application itself for the ones that are going to to be built uh which will install those requirements now for some reason I actually know why this rebuild did not happen and it’s because I did not do Docker compose watch so that’s something else I need to remember to do and that is Docker compose and up– watch so that it does rebuild it when requirements.txt changes so that error goes away but the key part about this is that now I have another place for my environment variables if I refresh on the docker composed version or my local version I should should be able to see the environment variables coming through for either one and now I have support for both places now the reason I’m doing this as well is so you get very familiar with the process of loading environment variables especially as you work towards going into production because you’re going to want to help isolate these things going forward now notice that this EMV do composed is underlined or it’s ready to be going into um you know git so if we actually look inside of our git step is here we can see that env. compose is in there if we look in the GitHub repo the actual sent code we can see thatv is not in here this is an example of I’m okay with sending this environment variables because composed yl is still using them and in this case it’s really just for a local test environment one that I consider disposable as then I can delete it any time obviously the usernames and passwords are not great here this also should indicate to you that the actual time scale service if you were to want to use compose in production this these environment variables should also be in an EnV file and be done differently inside of there if you were using compos and production which we are not going to be using at all we will be using Docker files but not Docker composed okay so that’s loading in the environment variables in both places as you can see Docker compose is far easier than what you would do what I just did but the other thing about this is beyond just having environment variables if you have any other static variables you want to set like other settings for this entire project you could also put those inside of this location as well now we’re going to convert our pantic schemas into SQL models this is before we actually store it into the SQL database I really just want to see how the feature of SQL model work in relation to the actual pantic models so what I want to do here is grab this schemas and I’m going to go ahead and copy it and make a new file called models so realistically what we’re doing here is just updating the previous ones to be based in SQL model and it’s going to be incredibly straightforward to do I’m actually going to name them the same thing for now we might change them later but for now we’ll leave them as event schema and so on mostly for the ports when we go to test this so the idea here is we want to not use pantic but rather use from SQL model we’re going to go ahead and import the SQL model and then also the field okay so base model is now going to be SQL model so we go ahead and replace all of these in here like so and I shouldn’t have to change the field but overall this is like the main change that I’ll need to do with the actual ID we will probably have to change that later which you can actually see in the documentation it’s going to look like this at some point so we might as well put that in there uh at least as a comment for now and then we’ll come back to that in a little bit uh but what we’ve got here is basically the same thing as we had before so inside of routing instead of schemas we’re going to go ahead and change this to simply just do models then we’ll go ahead and jump into our notebook and we really just want to test to see if all of the notebook stuff is still working let’s make sure our app is running I have it running in two places I think it looks like it is so uh inside of Docker of course and then in my local machine let’s go ahead and run this all together and everything works as it did prior um so we should have the actual models all set up as we had before okay so the big change from here would then be to change this into a table as in this is what we’re going to actually end up storing of course we’re not going to do that just yet the point of this was really to see that SQL model and pantic are the same and of course if you actually look in the documentation you will see that at the same time it’s also a pantic model so it’s quite literally a drop in replacement for this as we’ll see very soon it will also help us with storing this data now we need to create our first SQL table our database table the way we’re going to do this is a three-step process the first step is going to be connecting to the database itself using a database engine when I say connecting I mean having python being able to call the SQL table right that’s it that’s the connection that’s what database engines allow for us to do after we actually can connect to the actual database we need to decide which of our schemas in here is going to be an actual database table that might be all of them it might be one of them it might be none of them but the idea is we need to decide which one’s going to be a table and then make sure that that it’s set up the correct way to be a table and then finally inside of fast API we need to make sure that fast API is connected to all that so those tables are actually being created and they’re be being created correctly okay so the first thing that I want to do then is actually decide which of these are going to be database tables now you may have never worked with databases before or you have and you just didn’t realize how these tables work to make a decision like this so if you think of a database table very similar to like a spreadsheet you’re on the right track spreadsheet have all of those columns and you can keep adding columns as you see fit but then when you actually want to store data those columns describe what data you want to store in that particular column this is not that surprising you might have a column for ID you might have a column for page you might have a column for description all of those columns in a database need to have data types so in the case of an ID the entire column is only integers they’re only numbers in the case of a page those can all be strings and every once in a while you might have a datetime object or you might have float numbers or all sorts of different other data types there’s a lot of them out there the point is SQL tables especially inside of the structure of SQL have the ability to do very complex operations on all this data now when you think of a spreadsheet you might only have a few thousand rows in there at any given time maybe you have up to a million but if you got a million rows in a spreadsheet there’s a really low likelihood you’re going to be opening up in in Excel or something like that you probably need to use something different altogether but the idea here is a spreadsheet has limits to the number of rows SQL tables on the other hand basically have no limits or at least in many cases they’re designed to have massive amounts of data in there and so having these data types make a huge difference in how we can use those tables so for example if you want to get IDs that are greater than 100 or a, the actual SQL table itself will be able to do that very very efficiently versus trying to go one by one and looking for that specific ID there’s a lot of advantages of using a database itself to store data I’m not going to go through those advantages the point that we need to know right now is that our columns need to have specific data types now this is the cool thing about SQL model is we’ve already declared those data types integer string string right so we already have three data types in here which are going to be actual fields that will be in the database itself otherwise known as columns columns Fields you can think of them as the same thing in the case of how we use our SQL model so the idea here then is we want to decide which one of these is going to be a table now this is actually fairly straightforward if we look at how we’re using it by going into routing. PI right you could already have some intuition about which to pick already but let’s go ahead and go through each view to see how we’re using it and how this might affect what we decide to store in our database so if we look at the very first one we’ve got an event list schema this is a git we are only extracting data from the database that’s it so there’s a really good chance when you extract data you’re going to not store the extraction that you just did so that kind of is like a circular Loop if you were going to store every time you extracted something that doesn’t make sense now you might store the fact that it was extracted but you probably won’t exore the actual data that was extracted so this one I think the event list schema we can say yeah it’s probably not going to be a database table the next one create event now the incoming data is what we end up storing but do we always want to store all of that incoming data or do we want to make sure that the incoming data is flexible in other words this event create schema here if we were to change it let’s say for instance we didn’t want to store a description anymore on that crit event we want to remove that do we want to remove this field from the entire database itself the answer in that case would be no we would want to keep that field in there it’s just when we first get that data we aren’t going to want to um you know maybe have that field anymore okay so in that case this one we’re probably going to rule out but notice that it actually turns it into an event schema we return back the event schema itself now this is the one we’re definitely going to want to store so we get a payload coming in of data that’s all validated and all that then we want to turn it into a different data type which means that we’re going to end up storing it in inside of the event schema table which will change very soon now another thing about this is we notice that the event schema itself is used multiple times for database like things in other words if we are looking up that database table we see event schemas in there so that means it’s probably going to grab data from there and then also when we go to update data um we’ve got some data coming in that we might want to change and again we can decide which fields we want to actually allow for updating to happen without removing those fields from the database then we can actually return back that event schema so yeah event schema is the only one we really need to St save in here um and of course the event list view is doing a result of the event schema as well so that’s the one that’s going to be our database table it’s going to be this one I actually do not want to call it event schema anymore I now want to call it event model the reason is hopefully straightforward but let’s go ahead and actually modify this a couple times we’ll call it event model and then I want to go ahead and add in here table being true and then of course if I change this event model I’m also going to delete schemas dopy we’re just going to use models.py in this to keep it simple and then inside of routing we’re going to go ahead and do some changes in here because it’s no longer event schema it is now just event model so all the places that had event schema I’m going to change to event model now the reason I’m calling event model is you might as well think of it as event table or event database table and that’s kind of the point here so that’s why I’m going through the process of changing all of the places where it said event schema and now also when I look at the schema stuff I can kind of think of those in terms of not being a database table or not being a database model um and that’s kind of the point now these could be pantic models but in terms of the documentation from SQL model they recommend you use SQL model itself there’s probably advantages to it that we could obviously look up but for now we’ll just leave it as is now we need to connect python to SQL by actually creating a database engine instance and then actually create the tables themselves so inside of the DB module here we’re going to go ahead and create session. piy this will handle something else later which is why it’s called session but the idea here is we need to create the engine itself so engine equals to something and then we need to actually Define something that we will initialize the database itself which should be something like creating database okay so before I actually even create the engine itself we need to know when we’re going to run this init DB stuff now the idea here is in main.py this is where we will run it in what’s called a lifespan method for fast API but before I show you the lifespan method I wanted to show you the old way of doing it which is an event so if you actually came in here and did app.on event and something called startup you could then Define on startup and then this is where you could actually run the init method for DB this is still a valid method it’s still works but it is deprecated meaning it’s not going to be supported in the long run so we do something a little bit different than this now if you have these onevent stuff you’re going to want to either do all on event or none of them before you do the the context method or the context manager method which is what we’re going to do right now so what I want to come up at the very top we’re going to go ahead and bring in from context lib we’re going to import the async context manager this right here and then up here we’re going to go ahead and use that async context manager it is a decorator so we’ll go ahead and grab it and then we’re going to define the lifespan but it needs to be async Define lifespan and it takes in the app which is a instance of the fast API app so you could absolutely use the fast API app in here if you wanted to this is we’re going to where we’re going to go ahead and call in it TB and then we need to yield something and then this last part would be uh any sort of cleanup that you might want to do so basically like uh we’ve got here let’s remove this we’ll go ahead and say before app startup then it yields to actually run it and then we actually can clean up anything in here now I think the reason they brought this in was because of AI models and with AI models you might want to run this lifespan thing in here so it kind of gives us before and after inside of our app but the idea here is we can pass this in to fast API so that’s what we’re going to be doing now we want to actually bring in that in method here so we’ll go ahead and do from api. db. session we’re going to go ahead and bring in initdb now of course we actually need the database to work in here so we’ll go back into our session and create this stuff out so the first thing is going to be our database engine we’re going to import the SQL model uh itself not SQL Alchemy but SQL model and we’ll create the engine by doing SQL model. create engine and then passing in the database URL string as we see here so that also means that of course I need to go ahead and do from. config I’m going to import the database URL and we’ll go ahead and pass that in here now the important part about this database URL is it does have a default in here of an empty string so I want to actually make sure that that’s not the case so we’ll go ahead and say if database URL equals to that empty string then will raise a not implemented error saying the database URL needs to be set now the reason I’m defining the engine here altogether is because later we’re going to do something called get session which will handle or use that database engine as well so now that we’ve got that we need to actually initialize the database itself which is going to be from SQL model once again we’re going to now import the SQL model class and we’ll use this class to create everything so it’s SQL model. metadata. create all and it’s going to take in the engine as argument so now what this is going to do is obviously make the database connect to the database and make sure that that database has all of the tables that we’ve created in the case of table being true now one of the things that it will not do is it will not search your python modules for any sort of instance of this class right here that’s not how SQL model works it only actually creates those tables these tables right here when those models are actually being used in our case they are being used and routing. we actually have this the event model in here it’s being imported this routing is also being imported into our main application from event router in other words the actual SQL model metadata engine thing should actually work now and so let’s go ahead and save everything if you do save everything what you might get is an error like this where pyop G is not installed and then in you also might get it in both places actually so what we want to do is make sure that pyop g is installed right now I don’t actually have it installed in here so of course we’re going to do that in just a moment now if you remember back to environment variables we actually did this plus pyop G in here that’s part of the reason that it’s looking for it that’s why um you know we’ve got this error in altogether is it’s looking for that specifically so what we need to do then is inside of requirements.txt is we need use that I’m going to be using the binary version which is why I put that bracket in there as binary this is scop G3 or version three there is another one called pyop 2 and you would see that and it would be something like this I believe that’s the old version that you would end up using so now that we’ve got this requirements.txt of course if you’re using Docker which going to end up happening is this should actually rebuild everything looks like we’ve got a little bit of an issue going on there but if you’re not using Docker which I have both of them up for a reason you would then just go pip install d r requirements.txt and hit enter this should install all of it using pyop G binary I think is the way to go uh it makes it just a little bit easier to run the integration itself there are other ways to connect it we’ll talk about that in a second but I just want to make sure that this loads and it runs it looks like it’s loading and running but I get this error in here this is another error that we definitely need to fix and we’ll talk about in just a moment but of course on the Docker side it actually rebuilt everything and Docker looks like it is working just fine so we’ll come back to our local python in just a moment the key here though is really this lifespan in here and that it is creating these database tables now we won’t know if these are created just yet we need to solve that one error so let’s talk about what that error is and how it relates to Docker and also using Docker locally in that last part we saw an error where psychop G was not available it was not found and the reason that we saw that error in the first place has to do with how we were mapping in our database engine so if we look into our session here we see that we’ve got this create engine for this database URL the pyop G error happened because of our environment variables with this plus pyop G and we can actually see that in both places whether it’s in our local python project or in our Docker composed version of it so both of these showed us that specific error now the thing about the SQL model and more specifically the underlying technology which is SQL Alchemy is it can use more than just postgrad it can use MySQL it can use Maria DB and there’s probably others that it’s supports so this create engine here from a database URL is very very flexible which is why in ourv we have to specify postgress SQL plus the actual package we’re going to use from python to connect which is why it raised that error because we never actually installed it once we installed it that error went away but it brought us a new error which is related to connecting to that database now this is not really that clear as to this right here of course if you do a search or if you look at this it will probably show you that there’s something with a connection that’s failing and of course if we look at our Docker composed one that one is actually not failing it is working correctly or at least it seems like it’s working correctly so the reason that the python one is failing is because of where our database is coming from it’s coming from Docker compose this DB service right here so inside of the docker composed Network which has both of these apps they can communicate with one one another that’s one of the great things about using Docker compose is there’s internal networking inside of Docker compose which is why we were able to specify our database URL in this way so I could say DB service which is literally the name of the service and it would be able to connect it’s a little bit different than like going on our local browser right so when we go into our browser and look at our app it doesn’t say app here it says 00 Z right so if I tried to do app here I would get a connection error because that is not how my local machine can connect to this app it has to be on 0000 or my Local Host basically I should be able to connect with simply local host or at least I could try to connect with Local Host and that also seems to be working working just fine so the way we connect to our actual database service though is a little bit different than the way we would connect to our app itself now the reason it’s a little different is the app itself is connecting slightly different than our browser so if we actually look in the EnV here we see that it says host value if I change this to Local Host I can try that one and see if that ends up working as soon as I save it let me just restart the python application itself and see if that does it that seems to do the trick right and so if for some reason that was not doing the trick I want to show you one other thing we could try as well so let me close this out and inside of Docker compose I’m going to go ahead and get rid of this expose here for a second we’ll save that and we’ll run this again and it’s still connecting so we’re doing okay in terms of our ports are concerned but every once in a while you might need to go into EnV and change it from Local Host to being something based off of Docker itself which is host. doer. internal so that might not be another environment variable you would need to run in order for your app to be able to run as well but right now this is still also giving us an issue so that is not one that you’re going to want to use you’ll want to stick with Local Host just like our application has been using before so the reason that I wanted to show you the docker host internal one so I’ll go ahead and leave this commented out right above here uh the reason that that even exists is because you might want to try it from time to time depending on your host machine itself and how that ends up working so it’s host. doer. internal will often be mapped as well so this is of course one of the challenges with trying to use Docker compose without putting your app in there as well but to me I think developing this way is fairly straightforward but of course every once in a while you might need to use a database service that’s not inside of your internal Network itself uh so there’s a lot of different things that we could talk about there but the point here is we need to solve that single connection error now this is also more easily solved by using a actual you know database that is definitely going to work like if you went into time scale and created a database you should be able to actually bring in that connection string it might also still need to use this stuff right here because of how we need to connect but the actual host and all that stuff all of these things would then be based off of like time scale an actual production ready one that we could then go off of as well of course we will see that very soon but at this point now that we’ve solved some of the database issues we can move to the next part which is actually using our notebooks to see if we can actually even store this database stuff which actually takes a little bit more than just using those notebooks let’s take a look let’s store some data into our database the way this is going to work is first off we need to create our session function inside of the DB session module we’re going to go ahead and create something called get session here and then what it’s going to return back is with session session the session class itself which we’ll bring in from SQL model we’re going to go ahead and pass in the engine in here and then as session this is going to yield the session itself now the reason we have this is because then we can use it inside of our routes that need a database session so the idea then is grabbing that get session method so we’ll do from the api. db. session import get session we’re also going to go ahead and bring in depends from Fast API and then we’ll go ahead and bring in from SQL model not SQL Alchemy but from SQL model we’ll import the session as well so all of this allows us to then come into an event like this and say something like session and this is of the data type session and then it equals to depends and get session that’s it so this function right there will yield out the database session so we can do database session stuff down here now the thing about this function now is it’s getting a little bit bigger so I’m going to go ahead and separate things out a little bit and I’m also going to put up response model up here just like that so we don’t have to use that Arrow method anymore you could do that on all of them both methods are supported and work just fine okay great so now we’ve got this database session I want to actually use my model I actually want to store things in my model so the first thing that I want to do is I’m going to go ahead and say object object and that is going to be our event model then we’re going to go ahead and do model validate of the data that’s coming through so this data right here we’re going to pass into validate so we’re basically validating it again but more specifically to the model we’re storing it with then we’re going to go ahead and do session. add that data so when I say obj I think of this as an instance of this model or basically a row of that class but I like calling it obj it probably is an old habit you can call it what you’d like just make sure that it’s consistent throughout your entire project whenever you’re doing something like this and then you’re adding it into the database so this is preparing to add it then session. commit is actually adding it into the database then if we want to do something with this object as in get the ID we would do session. refresh of that object and then we would just return back that object itself because it’s going to be an instance of this model class itself and that’s how we do it that’s how we can add data into our database it’s really just that simple now of course it we have to remember that if we want to use a session this is the argument that has to come in there we use the session variable through here if I called this sesh that would be fine you would just want to update it down here that’s not common so definitely consider that when you change these names but the idea is of course it’s a session class and it depends on that function which will yield out a session based off of our database engine which is of course why I have that database engine up there in the first place so now that we’ve got this we’ve got a way to actually add in data which we can verify by going into our notebook and trying this out so I’m going to restart in my notebook here and I’ll just go ahead and run it and what we should see is there’s some data in here with an ID now do we recall whether or not that ID was valid before I don’t know let’s do it again there’s another ID and we do it again there’s another one and we do it again and another one so we’ll Contin continuously increment this ID even though we didn’t specify it directly inside of here all we are specifying on that API request is the data that’s coming through and so it’s automatically incrementing it for us thanks to the features of a SQL database in this case it’s a postcg database but we’re automatically incrementing with this being a primary key as I mentioned before and we can see that result right here when we go through and actually add in more data now of course we need to fill out the other API routes as well which we’ll do in just a moment but before we do that I will say every once in a while as soon as you start filling up the database with some of this test data you’re going to want to come in here and do Docker compose down with that- v to take down the database itself this will delete that database and that Network it alog together and then you would just bring it back up with that watch command just to make sure that everything’s changing as need be now the database is completely fresh which we can then test again by jumping back into that notebook and running the same commands now we get a connection error with the notebook which doesn’t make that it makes a whole lot of sense with how these sessions work within a notebook so I’m just going to restart the notebook and then I’ll go ahead and try that again and I’m still getting a connection error so that’s probably not good this is probably a little bit of an issue with one of these things so let’s go ahead and restart this application which Port are we using up here let’s see here we’ve got a 802 so everything should be back up and running again let’s see looks like the database is ready let’s go back into the notebook here and maybe I just went a little too fast for it and there we go so now it’s back into being connected with that one let’s try this one here and there we go so we should be good to go in terms of that data and there it is auto incrementing okay so every once in a while you might need to let things refresh and actually rebuild as well so saving data and doing all that will help flush out that process because you’re probably not going to bring the database down that often especially not while you’re talking about it we’re now going to ask our database for a list of items a certain number of items that we want to come back this is going to be known as a database query that will return multiple items so the first thing that we want to do is look into read events this is where we’re going to be changing things because well that’s our API inpoint to list things out now the first thing I want to do is bring in the session because I’m using the database so I definitely want to have this session in there to be able to use that as well now once again we can use that same response model idea up here as well but this time instead of the event model it’s going to be the event list schema which we of course already have format it out right down here so the idea is we want to replace this list right here with the actual results so the way we do this is by doing something called a query so we set it equal to a query now if you’re familiar with SQL you might be familiar with something like select all um and then from some sort of database table itself which in our case is the event model we’re basically doing that as well but we’re going to be doing it from the actual event model class in other words the actual SQL model om or uh so which means that we need to bring in this select call here which we will go ahead and do select from this event model and so that is a basic query that we can do then from there we can actually get the results by executing this in the database itself which is our database session. execute and then the query itself and then the final thing is we would do all this would allow us to get a bunch of items in here and which case I should be able to return them just like that the reason I should be able to return them is because of this model itself is inside of the list schema also right so going back into that schema itself we see that the results are looking for instances of that model which is exactly what we’re doing here we’re returning those instances in this request that’s what’s happening with that so that’s at the the Baseline what is going to happen and then we can also grab the length of those results and then there we go so this should give us some results in here now if we actually go back into our notebook for the list data types here I added this print statement in here so we can actually see it a little bit better which allows us to see hey there’s the ordering that’s going on there’s all of that data of course if we were to add in some more data we can see this again so I’m going to go ahead and add in a few more items in here and just run this cell several times to just make sure that I have maybe over 10 items in here as we can see by this ID or maybe even over 12 so later we can use that 12 number again so now that I’ve got 12 in here let’s go back into my list data types and run this again and we can see there’s my 12 in here so this becomes a little bit of an issue is our results here have 12 items but maybe we actually only want to return back 10 items in which case we would come back in a routing and we would need to update our query here by having something called liit and then we can pass in the amount that we might want to limit in terms of the result itself in which case I should be able to go in here and then I can run that limit itself and then there’s 10 as a result and we see that there’s only 10 items in here now I can also update this so the ordering or the display order is a little bit different as well so back into our routing here we’ve got this query still we can come in and then do something like order bu and we can add in an ordering in here now how does this ordering work well it’s going to be based off of the actual table itself which of course that table is our event model inside of that table we have different fields in there in my case I can use ID that field is this one right here we could use page your description as well but those are the only fields we can really order this by and then we can do descending which just changes the order it flips the order in here which we could verify by going back into our API and doing another request and we see now it’s flipped and ID of 12 is coming up first and two and one are no longer in the results altogether and of course if I wanted to flip it back I can change it to ascending and of course that will flip back that order with different values in here okay so this would be like okay the next level of this would be doing pagination or allowing to have only 10 results coming back but then multiple pages of 10 so you can you can do this request several times to get all of the data cuz realistically you would not get all of the data once you know you would probably never do this command that’s probably going to end up being too many things so you would often use a limit you might have a bigger limit like 100 or 500 or a th000 it really is going to be depending on your API capabilities but the idea here though is we do want to have a limit of some kind and by default you may have noticed that the actual ordering was based off of the ID it was actually going up like this it was doing that ascending from the ID um which is also very nice it’s a very easy thing to work with but that’s our list view now that we’ve got that we might want to make it even more specific as in getting a detail view a very specific item that we can work with and then maybe eventually update as well let’s take a look using SQL model we will do a g or 404 error for any given event now what we’re going to do here is very similar to what we’ve done before first off I’m going to use the response model again and this time it is just going to be the event model like that and then now we want to build out the session query itself so I’ll go ahead and grab the session data and we’ll bring this in as well now what we want to do of course based off of this model we want to look it up for this event here so we need to define the query first which is going to be our select call and it’s going to be based off of this model here and then we use something called where and we can grab that model do a field name and we can set it equal to this event ID that’s being passed through and that’s going to be our query now so then from that query we get a result which would be something like the actual data itself which of course is going to be our session. execute and then we’re going to grab that query and instead of using all we’re going to use first the value here would then be returned and that’s pretty much it for our detail view or is it it’s Clos close it’s not quite the whole story jumping into our list data types here we’re going to go ahead and copy this git event down here and I want to just change it ever so slightly so that our path is going to be updated to be more appropriate for a detail view so I’ll call this detail path and that’s going to take in you know something like that 12 and then we’ll use the detail endpoint and I probably don’t need to put that base view in here anymore okay so now we’ll go ahead and grab that detail uh endpoint here for our request and I’ll just go ahead and set this to R and then we will do R all across the board and once again PR print this time instead of status R okay we’ll do our status code in here and see what that looks like okay so if we look at our list view we should still see uh that there is probably 12 in here right so let’s go ahead and just change our list view real quick to descending just to make sure that we have some data in here that is accurate to what’re looking up okay so back in here and list view there we go so we got an ID of 12 I’m going to go ahead and run this and there we go so we get that data of 12 if I go one Higher I get a 500 error that is a server error this should not raise a server error it should raise a different kind of error so this is not quite done and it has or rather this is not quite done and it has to do with this right here this query res returned back a none value and we can actually see the error in here that we get the input is simply none right because we’re trying to return back the event model but we’re actually returning back none so really what we need to do is say if not result then we need to raise something called the HTTP exception and it’s Capital HTP exception and we want to add in a status code in here of 404 and then some sort of detail saying that this item was not found or event not found right and so we need to bring in this HTTP exception from Fast API we’ll bring this in like that save it and now we’ll go ahead and try out this same sort of lookup and we’ll do it again now we get a 404 the actual error that you should have for a item that’s not found not found in the database in this case and so the cool thing about this is the same sort of idea if I made a mistake on this lookup and turned this into let’s say a string value here and and still did the lookup let’s take a look at what happens there 500 error this time so let’s go ahead and see what that error is right so it’s looking for the parameter with the ID of 13 so in this case it’s actually not looking up the ID uh based off of the string value right it’s not going to do that it’s going to give us another kind of error which shows us yet another time that hey you want to make sure that you’re using the correct data types when you do this lookup now the important part about the actual lookup itself the request that I’m making this is a string it’s just fast API parsing it into a number so that’s also another nice thing about this um but overall we now have our git lookup uh it’s fairly straightforward the next part is using this same sort of concept to actually update the data which kind of puts it all together the delete method I will leave to you it’s actually fairly straightforward and there is plenty of documentation on how to delete events which is something I’m not really concerned about at this point but updating one is a good thing to know because it kind of combines everything with we’ve done so far let’s take a look let’s take a look at how we can update the event based off of the payload that’s coming through keep in mind the payload is only going to have a certain number of fields the way we have it is literally the description that’s the only thing we’re going to be able to change in this update event which is nice because it limits the scope as to what you might potentially change so the idea here is we’re going to go ahead and grab the query and all of this stuff just like we did in the G event I want this to happen before I go to grab the payload data CU I don’t need it if I don’t have the correct ID now of course we still need the session in here as well and then of course I also want to bring in my event model as the response model to make it a little bit easier as well so I’ll go ahead and copy that and paste in here there we go and then I will update the arguments here just a little bit so it’s also easier to review as we go forward okay so let’s let’s go ahead and get rid of this now now I’ve got my session I can still look everything up as we saw before and this is now going to be our object this is really what it is I had it as result up here but it’s the same thing it’s still an object just like when we created that data so in other words we still need to do basically this same method here just like that and then when it’s all said done we will go ahead and return back that object so the key thing here though is the data itself how do we actually update the object based off of this data itself the way you do that is by saying for key value in data. items this of course is unpacking those things we can do set attribute and we can set attribute for that object of that key value pair that’s in there and again the actual incoming payload is only going to have a certain number of keys if for some reason you didn’t want to support those keys you could set that here so say for instance if k equals to ID then you could go ahead say something like continue because you’re not going to want to change that field but again you shouldn’t have to do this at all it’s a bit redundant based off of the actual schema itself but if you were to make a mistake then yeah of course you might want to do it then okay so with this in mind I should be able to run this and get it going so the idea here then is just checking this out or testing it out so if we look into the send data to API we have the API endpoint for this which is basically what we’ve done before so now we’ve got the event data here’s the description that we want to bring in this time I’m going to go ahead and say inline test and we’ll go ahead and run that now we’ve got inline test in here if I go back and list out that data let’s go ahead and do that and see what the description is now and there is that inline test so we were able to actually update that data as we saw uh just like that okay so this process is very straightforward of course if we tried to update data for an item that does not exist we will get a 404 right so going back into the send data here I’m just going to change it to 120 now we get a you know event not found and if of course if I change this response to status code we should see a 404 in here coming through great so yeah the actual API endpoint is now working in terms of updating that data again this one we probably won’t use that often but it’s still important to know how to update data in the case of an API not just with what our project is doing right and of course the challenge I want to leave you with is really just doing the delete method yourself self which is actually pretty straightforward given some of this data in here as well so at this point we now have update we have create and we also have list so we’re in a really good spot in terms of our API the last part of this section what we want to do is adding a new field in this case I’m going to be adding a timestamp to our model to the event itself having an ID is nice because it will autoincrement but actually having a timestamp will give us a little bit more insight as to when something actually happens now what we’re doing here is we are actually going to delete our previous models and then go ahead and bring things up in other words I’m going to come into my Docker compose down and then- V this will of course delete the database so it’s a lot easier to make changes to my database table so what you would want to do in the long run is use something like a limic which will allow you to do database migrations it will like quite literally allow you to alter your database table but in our case we’re not going to be doing that because once we have our model all well organized we should be good to go and not need to change it very often and if you do running this down command is not really that big of a deal altogether now before I start this up again and this is going to be true for any version of this app I want to build out the actual field that I’m going to do right now so the idea here is we’re going to go ahead and do something like created at and this is going to be a date time object so we’ll go ahead and bring in from date time we’re going to import date time okay and this is going to be equal to a field here now this field takes a few other arguments there’s something called a default Factory this default Factory has to do with SQL alchemy that will allow me to do something like Define get UTC now and then I can return back the date time. now and then we can actually bring in the time zone in here and do time zone do and UTC and then I can just go ahead and say um that this value should be replaced for the TZ info of time zone. UTC okay so just making sure that we are getting UTC now this will be our default Factory so when it’s created it’s going to hopefully go off of that function and work correctly now when it comes to the actual database table there’s another thing that we need to add in this is going to be our SQL Alchemy type the actual database type itself which is going to take in from SQL model which we need to import SQL model as well and oops that’s not what we wanted so we’ll do SQL model here and then I’ll go ahead and bring in SQL model. dat time and then this is going to be time zone being true so when it comes to SQL Alchemy you need to Define things for datetime objects it’s definitely a little bit different than what we have here but the SQL Alchemy stuff does creep in a little bit with SQL model when you’re doing a little bit more advanced fields datetime is one of such field and then we’re also going to go ahead and say nullable being false okay so doing nullable being false is yet another reason to take down the database and then bring it back up so at this point we should have this field automatically being created for us when we go forward okay so now I’m going to go ahead and run the Ducker compos up again and this of course should create the database tables for me and I’ll also go ahead and run my local version of the Python app as well which is attempting to create those database tables as well so the only real way I can test to make sure this is working at least how we have it right now is to create new data so we’ll go back into this notebook that will allow for it and hopefully everything’s up and running as it was looks like I’ve got a connection aborted so maybe I need to do a quick save on one of my models just to make sure that that’s up and running looks like it is now so let’s go ahead and try that one more time we definitely saw this before so I’ll go ahead and run it again and it looks like that time it worked okay great and so now if I go through this process I can see there is a time zone being created in here and of course I could do this as many times as necessary to update future Fields if I wanted to which in my case I’ll just go and go to five and notice that created at is in there as well now one of the other things that you might consider using this same idea of creat at is updated at so you could add a whole another field that’s very similar to this but it won’t have a default Factory uh necessarily you could still leave a default Factory but if you wanted to create it at field or an updated at field in your update you know command here you would then go ahead and set it again so you would say something like updated at and this is where you would go ahead and be able to set that field itself for that time stamp when it may have been updated that’s outside the scope of what we wanted to do here but the point is I wanted to make sure that I have a bunch of different fields in here that make a whole lot of sense in terms of actually storing data especially when it comes

    to a datetime field like this we need to see how we can actually store individual objects itself um at any given time and also have a field that is automatically generating inside of our database which is exactly the point of this whole thing let’s actually take a look at how we can do that updated field I’m going to go ahead and come into our event model here we’re going to paste this and change it to updated at just like that there shouldn’t be a whole lot of changes we would need to that but of course this also means that I need to go ahead and do Docker compose down of that actual entire thing so we can bring it down now back into our routing what I want to do is I’m going to bring in the git UTC now method here so it’s using the same functionality as when it’s being created so that it grabs what that value would end up being based off of all that data so then in our update view then we come in here and I’m going to go ahead and bring in object. updated at and then this is going to be the G UTC now and then that should actually set it and get it ready for when we go and store it okay great so with this in mind I’m going to go ahead and run this again we’ll bring it back up now one of the things the reason that we have it with the same default Factory is because when it’s initially created we might might as well also set the updated at field at the same time so they’re going to be the same value when they’re initially created but then in the future they might change if you then update them and let’s just make sure that everything is running looks like it’s all running we’re good to go so let’s go ahead and give this a shot so first and foremost we want to just send some data in which I will do with that post method right there and there we go so we’ve got an ID of one in here I’m going to change this to an ID of one as well and we should see that updated at and created at are the same time um it might be slightly different because of the microsc but overall the actual seconds are the same as we can see right here and right here great so now I’m going to go ahead and update it and we should now see a change the created at should be the same the updated AD should be slightly different and it’s not really that much different but it is different it’s different enough to see that that actually ends up working so that’s how you end up going about doing that now of course you could also have a field that is completely blank you don’t necessarily have to automatically update it uh in the future but that’s just a simple way to make that change so you can really see those time differences uh if you wanted to now the other nice thing about this too is then in our list view what we could do is we can come in here and do updated at and go based off of that value instead of we could also do created at of course but now we can go off of a different value alog together in which case we can come back in here and we can see the different data items that are coming through in here which we only have one right now so let’s go ahead and send a couple more and of course uh we’ll update the first one again just to make these things a little bit out of order and so let’s go ahead and try that again and what we see in here is now the different data is coming through on how it ends up looking the updated at is the first one because of course that’s how we defined it but I can always re flip it if I wanted to based off of the routing itself so now ascending so the oldest one being first we can go ahead and take a look again and let’s list that stuff out and now we’ve got the oldest one being first the most the the oldest updated one not the oldest created one uh being first and then the most recent created one being last or updated one being last again okay so cool um very straightforward uh I think that would have been really easy for you to do off the video but I just wanted to show you some techniques to do it in case you were curious about it and most importantly to show you this ordering stuff as well as to you can use updated and created app we now know how to store data into a SQL database thanks to SQL model it makes it really straightforward to use this structured data that is validated through pantic and then actually stored in a very similar way as it’s validated which I think is super nice and then extracting that data really looks a lot closer to what the actual SQL is under the hood which I think will actually help you learn SQL as well if you are trying to learn it because the way this query ends up being designed designed is it’s very much like SQL itself so SQL model I think is a really really nice addition to the python ecosystem and of course I’m not alone in that it’s a very popular tool but the idea here is we have the foundation to read and write data into our database now we need to elevate it so we can read and write a lot more data and do so by leveraging time series when it comes to analytics and events specifically we definitely want to turn it into time series data now the difference between it are going to be mostly minor based off of all of the tooling that’s out there so actually turning this into time series data is very straightforward so let’s go ahead and jump into a section dedicated directly to that we’re building an analytics API so that we can see how any of our web applications are performing over time in other words our other web applications will be able to send data to this analytics API so we’ll be able to analyze it and see how it’s doing over time the key word is overtime there now a big part of the reason that our event model didn’t have any time Fields until later was really to highlight the fact that postgres is not optimized for time series data so we need to change we need to do something different and of course that change is going to be time scale and specifically the hyper taes that time scale provides now we have been using a time scale database this whole time but we’ve been only using the postgress portion of it so now we’re going to convert it into a hypertable because that revolves around Time and Time series data it’s optimized for it now a big part of that optimization is also removing old data that’s just not relevant anymore now it removes it automatically and of course you can build out tooling to ensure that you’re grabbing that data if you wanted to keep it in some sort of Cold Storage long term but the point of this is that we want to optimize analytics API around time series data so we can ingest a lot of data and then we’ll be able to query it in a way that we’re hopefully kind of familiar with at this point by using postc so at this time we are now going to be implementing time scale and specifically a package I created called time scale python we’ll look at this in just a moment but as you can see here this is exactly like our SQL model with a few extra items a few extra steps to really optimize it for time scale DB now these extra steps are not really that big of a deal as you’ll come to see and a big part of the reason they’re not a big deal is because the time scale extension is the one that’s doing the heavy lifting this is just some configuration to make sure that it’s working let’s go ahead and get it started right now we’re now going to convert our event model which is based in SQL model into a hyper table or at least start that process so we can start leveraging some of the things that time scale does super well so the way we’re going to do this is we are going to go ahead and bring in from time scale DB we’re going to import the time scale model and then we’re going to swap out SQL model for that time scale model at this point it is still mostly the same so if we actually look at the definition of the time scale model by hitting control click or right click you can see the definition of this model it is still a squel model it is based off of that and then we’ve got ID and time in here so two default fields that are coming through and then some configuration for time scale specifically but let’s take a look at these two Fields we’ve got an ID field hey that looks really really familiar to what we were just doing with our ID field it has another argument in here that is related to SQL Alchemy on SQL model if you ever see sa like that that is referring to SQL Alchemy which is a another part or dependency of SQL model um it’s an older version of SQL model if you want to think of it that way now the other thing here is is we’ve got a date time field which hey what do you know we’ve got a default Factory of get UC UTC now it is a datetime object with the time zone and it’s a primary key as well so we actually have two primary keys so the hyper tables which is what we’re building is going to rely on a Time Field absolutely which is part of the reason we declare the time field altogether the other thing is we have a composite primary key which actually uses both of these fields as the primary key so the idea here then is we want to actually use this time field instead of the one we have so if we look back in our event model we had this created at field in here which we can now get rid of as well as this ID field now the other thing you’ll notice is we’ve got this get UTC now it’s in both places and so we can actually use the time scale DB version of get UTC now instead of having our own now of course you could still have your own but it’s not necessary at this point since we have this third party package that we are relying on we can use other fields to rely on it as well so the last real thing about this model is deciding what is it that we’re actually tracking here now in my case what I’m trying to track or the time series data that I’m looking for is Page visits so how would we think about page visits well that’s going to count up the number of visits at any given time right that’s the thing that’s what we’re counting up this is a little bit different than like sensor data for example so if this was sensor data you would have a sensor ID instead of a page which would be an integer and then you would have a value which might be a float right and so you’re going to then do something like the um you know value or maybe let’s say the average value of a sensor at any given time right so that would combine those two Fields but in our case we want page visits we want to count up the number of rows that have a specific page on it so the reason I’m bringing this up is mostly so that when we think about designing this model we want to rethink What fields are required in the case of page that one is absolutely required we we definitely need page otherwise it’s not an event so we’re going to go ahead and say that it is a string and this time we’re going to go ahead and give it a field with an index being true now making an index makes it a little bit more efficient to query but the idea here of course is going to be it’s going to be our about page or our you know contact page or so on and of course inside of time scale or postc in general we can aggregate this data we’ll be able to count the pages we’ll be able to do all sorts of queries like that which we’ll see very soon but the rest of the actual event model we could either get rid of or we can still use the updated at I’m probably never going to update this data but it is nice to have that in there and then a description in there as well so we’re really close to having this as a full on hypertable it’s not quite done yet and we still have to modify a couple things those are related to how we are using our session so we’re going to change our default engine in here and then we need to run one more command related to initializing all of the hyper tables it’s time to create hyper tables now if we did nothing else at this point and still use this time scale model it would not actually be optimized for time series data we need to do one more step to make that happen a hypertable is a postgress table that’s optimized for that time series data with the chunking and also the automatic retention policy where it’ll delete things later this is a little bit different than a standard postgres model but again if we don’t actually change anything it will just be a standard postgres model now we want to actually change things we want to commit those changes now the way we’re going to do this is inside of session. high we’re going to go ahead and import the time scale scale DB package in here and underneath in a DB I’m going to go ahead and print this out and say creating something like hyper taes just like that and it’s as simple as doing timescale db. metadata. create all engine this function here is going to look for all of these time scale model classes and it will go ahead and create the hyper tabls from them if they don’t already exist it’s really just a conversion process it’s still a postc model under the hood and then it actually turns into a hypertable now there are issues that could come up with migrating your tables so I’m not really going to talk about that right now it’s kind of outside the scope of this series if you’re going to be changing your tables very often but the point is that this right here automatically creates hyper tables the time scale DB package has a way to manually create them as well especially when it comes to doing migrations and making those changes that a little bit more advanced now the big reason that I’m talking about that at all is because we want to make sure our model is really good Ready Set because we probably don’t want to be changing our time scale model very often or really our hypertable very often unlike your standard postgres table you might change that more often than something like a hypertable now the idea here then is after we’ve got our creating hyper tables we have one more step that we need to do and that’s changing our create engine because we actually want a time zone in here as well so there is an argument that you can pass in time zone whichever time zone you are using in our case we’ve been using the UTC time zone which of course we see in our model itself where it’s get UTC now that’s going to be the default time zone we use and it’s the default one for the time scale model anyway now I recommend just sticking with that time zone because you can always change a time zone later using UTC makes it really easy to remember hey my entire Project’s in UTC we can convert it if we need to okay so the idea also with this time zone is we might want to put it into our config.py here which in which case I would go ahead and do something like DB time zone and then just paste that in here and then have the default being UTC once again in which case I would then import it into my session. and do something like that great so we’re not quite ready to run this command and when I say run this command I mean that we need to take down our old database and then bring it back back up so we can make sure that everything’s working correctly including any changes that I made to this model but there are a few other things that we need to discuss before we finalize this model and then has to do with some of the configuration that we want to use inside of our hypertable itself there’s a couple configuration items we want to think about before we finalize our hyper tables so let’s take a look at the time scale model itself and if you scroll down there’s going to be configuration items in here maybe some aren’t even showing yet the main three that I want to look at are these three first is the time column this is defaulting to the time field itself right so we probably don’t ever want to change the time colum itself although it is there it is supported if you need to for some reason in time scale in general what you’ll often see the time column that’s being used is going to be named time or the datetime object itself so that one’s I’m not going to change the ones that we’re going to look at are the chunk time interval and then drop after so let’s go ahead and copy these two items and bring them into our event model and I want to just go ahead and set these as a default of empty string here so what happens with a hypertable is it needs to efficiently store a lot of data which means that it’s going to store it in a chunk each chunk is like its own table itself now you’re not going to have to worry about those things time scale will worry about them for us the things we need to worry about are the interval we are going to create chunks for and then how long we want to store those chunks for in this drop after so this is actually very well Illustrated inside of the time scale docks by taking a look at a normal table and then taking a look at a hypertable a hypertable is a table and then it has chunks within that so as we see here we’ve got chunk one 2 and three and then each chunk corresponds to the interval that we set so for example this chunk is for January 2nd this chunk is for January 3rd and so on because the time interval is for one day the entire chunk is a day at a time that’s kind of the point here hopefully that makes a little bit of sense but the point of this also with respect to chunks is if you have a lot of data how you handle your chunks is going to be different if you have a little data right if there’s very little data your chunks could probably be a lot longer because you’re not taking up that much room in any given chunk and also you’re probably not going to be analyzing that data much for any given chunk so for example if this is your very first website your chunk time interval might be 30 days and it actually is going to be listed out as interval 30 days like that and so that’s how you write it you could say something like 30 days now one of the downsides of having an interval of 30 days or too long of an interval is you will have to wait after that interval passes to make a change so say for instance right now we say hey it’s going to be interval of 30 days that means all of the data coming in right now has a chunk time of 30 days if in a day or a week I want to change it down to one day then we have to wait that full 30 days before this new interval would end up changing now these automatic changes right now are not currently supported by time scale DB’s python package or the one that I created at least maybe at some point it will be but the point of this is we want to decide this early on I actually think interval of one day is pretty good for what we’re going to end up doing if we start end up having a lot of data then you might want to change this interval to like 12 hours but more than likely I would imagine one day is going to be plenty for us so the next question comes when do we want to drop this data now time scale will automatically drop this data based off of what we put in this drop after so in the case of our interval here we could absolutely say drop after 3 days but that actually doesn’t give us much for time for analyzing our data and what do I mean by drop after I mean quite literally after 3 days this thing would be deleted this would be completely removed from our table it’s not going to be there anymore this helps make things more uh optimized and also improve our performance altogether while also removing like storage that we just don’t need anymore we might not need that anymore so we could always take that old data and put it into something like cold storage like S3 storage but what we want to think about here though is when realistically would we want to drop some of this old data again it’s going to be a function of how much data we’re actually storing so this interval can also change as well so for example if I did two months then I would only have at most the trailing two months data if I wanted to have a little bit longer than that like 6 months then again I would only have at most 6 months of data at least in terms of querying this specific time scale model again we can always take out those chunks and store them later if we wanted to in my case I’m going to go ahead and just leave it in as three months I think that’s more than enough you might also change it to one month and so on um and you can change these things later but I wanted to make sure that they’re there right now so that we have them and we understand kind of what’s going on with our hyper tables or hopefully a lot more as to what’s going on with our hyper tables so the way we’re we’re going to be chunking our data is going to be with one day and we’re going to keep the trailing one day for 3 months so we’ll have at least that much data going forward let’s go ahead and verify that our hyper tables are being created first and foremost I’m going to go into session. here and I’m going to go ahead and comment out the process of creating those hyper tabls because we’re going to do it in just a moment then I’m going to go ahead and run Docker compose down just like that and then we’re going to go ahead and bring it back up so that we can quite literally have this working once again okay so we should see creating database regardless of how you end up doing it but the idea here is we now have our database up and running so to verify this what we’re going to be using is something called popsql or popsql decom and just sign up for an account it’s free especially what we’re doing and you’re going to want to download it which is going to be available on this desktop version right here and you can go ahead and download it for it your machine once you do you’re going to open up and you’re going to log in you’re going to probably see something like this so the idea here is you’re going to select your user and click on connections and then you’re going to create a new connection so we’ve got a lot of different connection types in here that we can use to really just run different queries that we might want to in our case right now we’re just going to go ahead and verify that our database is a hypertable database or has a hypertable in it so that means that we’re going to be using time scale here now the idea here is we could also use post credits both of them are basically the same right the connection type is basically the same it’s just a little bit different so we want to use time scale to verify that it is a hypertable when we look at it and then the next part is going to be our connection type we’re going to go ahead and use directly from our computer we want to make sure we do that and of course this is going to be on Local Host and the port here is 5432 the standard postest port and it’s also the same port we defined in composed. yl this port right here the first one of the two great so now we’ve got that we now need to fill out the rest of it the database name which of course is our time scale DB the username which is going to be our time scale user and then also the password time PW and that’s it then I’m going to go ahead and save that and we can run connect and now what we should be able to do is grab this data so I’m going to refresh here and we can see there’s our event model and I should be able to go to the event model itself and we can see that it’s not a hypertable at least not yet so back into our session here I’m going to go ahead and change it to see if it will create those hyper tables from here so as soon as I save this I should see it says creating hyper tables and hopefully I don’t see any other errors this is true regardless of what version we’re using here uh but ideally speaking we have all of our hyper tables so if I refresh in here this might actually show me my hyper taes it might not uh so the reason that it might not be showing our hyper tables is maybe we need to just start over a little bit in which case I’m going to go ahead and go Docker compose down again and then I’m going to go ahead and do Docker compose up just to see if that ends up solving it it might be a caching thing as well so we would also look at that but there we go creating hyper tables and all that let’s go ahead and refresh in here right now it’s still saying no which it shouldn’t so let’s go ahead and close out popsql for a moment and see if we can just reopen this up again I think it’s a caching thing which is not that big of a deal at all we just need to refresh pop SQL itself so that we’ll be able to run with our previous connection now there’s a chance that we might need to reenter our credentials I don’t think that’s going to happen uh but here we go and now it shows sure enough it shows a hypertable right away and notice that compression is disabled currently but it does show us the number of chunks we have so what if we actually start making some data the way we’re going to do this is by jumping into our notebook here I went ahead and had this one single event right so we had this post event that we did before off the video I went ahead and did this event count where it sort of randomizes stuff but it creates 10,000 events for us so I’m going to go ahead and run that and what we should see in our t uh table at some point it will just be one chunk uh so it’s going to take a moment maybe for that to be flushed out completely or maybe we need to do a query alt together I’m not actually sure um in terms of why that’s not showing up just yet but it is bringing in all sorts of data for us which is pretty cool so we will’ll test this out a little bit more the key of this was just to verify that it does say hypertable notice that compression is disabled that goes back into our time scale model where the default for compression is false so you can absolutely add in compression to make the data even more optimized for storage uh but for now we’re just going to leave it as is and then we’ll go through the process of actually seeing all of this data we will use uh popsql a little bit more to run some SQL queries or really to verify them but we’re not going to run that many SQL queries instead we will do the related queries right inside of our notebooks using um you know SQL model as well as time scale DB the python packages now what would be best is if we actually ran this a number of times over a number of days to simulate the you know actual process of visits but of course I can still run this a lot and we get you know 10,000 visits per time or 10,000 events per time and we’d actually be able to see a lot of data coming through and here uh at any given moment right so 10,000 events and of course this is happening synchronously so it’s not quite like what a real web application would be like uh but it is still at least giving us a bunch of data that we’ll be able to play around with let’s take a look at how we can do some of the querying with the SQL model now that we have a hyper table and we are able to successfully send data to it I want to actually be able to query this table and I want to do so in a way that is still leveraging the event model itself before I create the API endpoints now we have already seen how we can query it in fast API but I want to actually design the queries outside of fast API before I make it an API endpoint itself just to make sure I’m doing it correctly now the way we’re going to do this is inside of a Jupiter notebook so we’ll go ahead and do NBS I’ll go ahe and do five and we’ll go ahead and do SQL model queries. IPython notebook and then in here I’m going to go ahead and import a few things the first thing is going to be S the next thing is going to be from pathlib we’re going to go ahe and import our path here so when it comes to being able to import something from a model like models.py there’s no real clean way to do this I can’t just go from source. api. events. Models import event model right this will most likely not work because there’s no module named SRC now this is in part because we’re inside of the NBS folder here and so there’s a number of different ways on how we would be able to import this or grab it but the way I’m going to do it is I’m going to say my SRC path equals to path of dot dot slsrc and then we’ll just go ahead and resolve that right and so if we resolve that and take a look at what it is that is that SRC folder right there and so what I want to do is I want to pin this to my system path so all I can do is sy. path. append will take that string of that SRC path here just like that okay so that will append it to our system path which means then I can actually just go ahead and do instead of SRC I can just use api. events. models and I should now be able to grab that event model and have some no errors related to it and I mean I could also just go through and manually add all of this stuff in as well into the notebook itself but having this event model allows for me to do a few other things as well like I can go ahead and do from api. db. session we can import the engine as well which of course is going to give us this engine right here which has the database URL already on it or at least in theory it does so that’s another part of that as well let’s go ah a and run that one sure enough it ends up working so next up what I want to do is I’m just going to do a very basic lookup a very basic query for this first part and then we’ll come back and start doing more and more advanced queries so what we want to do here I’m going to actually put all of the Imports in the same sort of area we’ll go ahead and bring in from SQL model we’re going to bring in the session itself and then run that cell and then to use this session we’ll do with session and then we’ll pass in the engine in here and then we’ll go ahead and say as session and then I’ll be able to do my different session things that we might want to this of course is going to be basically the same as what we’re doing in our rep it just fast API handles it for us with that session stuff and all that but we can actually do the same exact query in there so let’s go ahead and grab that and just verify this in here with our session and I should be able to print out the results in here as well um we also need to import the select so let’s go and do that as well and now we’ll run this and we should get the same sort of data coming back okay so that gives us a result but what I really want to see right now is before we build out the rest of the queries I want to actually see the raw SQL query here now what we do is we bring in compiled query equals to query. compile and then we want to go ahead and bring in something called compile uh keyword RS here which I’ll just go ahead and copy and paste but the idea is compile keyword ARS the literal binds being true I’ll show you why right now so if we print out the compiled query and then if we print out the string of the actual query we see a couple different things so I’m going to go ahead and print this one little line to separate a little bit so the compiled query query is this one right here the other query has a parameter in here that does not have the literal binds in here at all so the nice thing about this is we would be able to change the limit with that parameter uh this is not something we’re going to do right now the point is I want to be able to use this compiled query by going into popsql I open up a new tab in here and I can paste that query in and just run that and so what it’ll do is it’ll run everything that we have in here with all of that data so we’ve got the pricing data and all the pages and all that so notice that the pricing data looks like we when we import put it in it didn’t have a initial slash in there which is kind of interesting uh but overall the data is in there and there’s a lot of data going on and this is how we’ll be able to verify that our queries are good and so we can continue to refine what kind of queries we’ll be end up doing inside of our database now when you’re working with database technology it’s a really good idea to do some of these queries just to verify that it’s working before you were to actually put it somewhere else so now that we’ve got that I want to jump over into doing time buckets we’re now going to take a look at time buckets time buckets allow us to aggregate data over a Time interval so it’s actually a really nice way to select certain data that falls into an interval in our case we’re going to end up counting the correct data for the correct pages but we want to start with just the basics of the actual time bucket itself so one of the things I want to do is grab this session here because we will be using the session itself and we’re going to be working based off of roughly this and so the nice thing about these queries here is we can put it into a parenthesis and of course this is just python related but then we can sep create them out line by line now I don’t really care about the limit right now so I’m going to go ahead and get rid of that and I’m not actually going to run this query we are going to chunk it down a little bit by using the time buckets so to do this we’re going to go ahead and do from time scale db. hyperfunctions we’re going to go a and import the time bucket idea here and then we’re going to go ahead and add in bucket equals to time bucket and then we’ll add in some amount of time I’ll leave it in as one single day now we want to do it on a specific field that we’re going to end up using which in our case is that event model. time now this is where keeping the actual field the time field consistent makes it a little bit easier so if I need to change to a different model time is still that time so that’s kind of the idea here so we can actually select it based off of that bucket itself we can come in here and just select that bucket we can also add in our page in there as well so if I do event model and then page I can bring that one as well that of course is an arbitrary field at this point but now what we can do is actually see this so I will leave that compile query because I might look at it in a little bit but for now I’ll go ahead and do results equals to session and then execute of the original query and then we’ll go ahead and do fetch all to see what that looks like and I’ll use prettyprint to actually do this so let’s go ahead and import prettyprint so from prettyprint or PR print import PR print okay so I run that and now I can see all of this data coming through based off of time and all that so there’s a lot of data I’m actually not sure how much data is in there it’s not quite what I want though right so I want to change this a little bit and of course we don’t order it by updated at that’s not even being selected we’ve got our time bucket and then our page those are the only things that are being selected and if we scroll down we will see that there is all instances of that so like it’s going to have duplicates in here but it actually removed the original time to just being based off of the date right it’s not quite the same as what you would see up here which of course we could verify by running the results up there as well so if I came up here and ran those results I should be able to see that there are uh differences in the time itself so the datetime object in here is certainly different than this so already showing the aggregations at work is kind of the point I’m going to go ahead and move these import statements up a little bit uh but now that we’ve got that out of the way we’ve got a way to gate this data based off of buckets we want to do one more thing and that is I want to actually count the number of items in that time bucket or based off of that time bucket so the way we do that is we bring in from SQL Alchemy we’re going to bring in a function in here and so this function right next to the page is just going to be function. count and that’s it we can give it a label as well if we wanted to we can say something like event count in here and then we could reuse that label later but I’m just going to leave it as simple as possible by using function count if I go ahead and run this we get this other issue now so this is now not necessarily giving us all of the right data so I’m going to go ahead and get rid of this order by here altogether we’ll just use the select for now still not giving us exactly what we want now this is happening in part because we need to group the data somehow so how is this count going to work what is it actually counting so the way we do that then is we come in here and do group by and the way we want to group this is with the time and the page so those two things together will give us the grouping that we’re trying to count so I can go ahead and run this again now it’s showing me the grouping that we’re trying to count this is now looking pretty good I think so it’s giving us all of this data in here and what if we change this to 1 hour we should be able to see something very similar to that maybe we didn’t change very much in an hour let’s change it to one minute now we can see those chunkin coming through uh because of all that data that’s been coming through so if I were to run and send more data in it’s been more than a minute um I should be able to see a bunch of data coming through with these chunks as well we could always take this a little bit further as well but overall what we’ve got is now a Time bucket chunking and we probably also want to maybe order it so we’ll do order bu and in this case I’ll go ahead and say bucket and then we will maybe also do the page itself so the page model is being ordered and then uh or maybe not even in that ordering we can play around with these things a little bit oh we forgot a comma let’s run that again there we go and so now it’s in order right and we see and of course it will skip minutes right so if there’s no data in there it’s not going to have any chunks so it’s going to be skipping things which is what we’re seeing here now there is a way to do something called a gap fill which is definitely a function that the time scale DB package has uh so instead of using time bucket there’s time bucket Gap fill which allows you to fill in empty spots that are missing that might be missing uh by aggregating the data in my case I don’t actually want any empty data coming in here I’m just going to leave it as is because these are the actual data points and the actual counts I’m not going to guess what the counts might end up being now of course the actual analytics for a page would be a little bit different than this if we were actually grabbing web traffic right it’s not going to be consistently skipping minutes based off of the ingested data for all of the data it’ll just be for some of it okay so another thing about this is we can then narrow this down to the pages we want in here so for example if we go back into when we created this data I’m going to scroll up here I’ve got a few pages in here so I can go ahead and grab that and we can grab those pages and put them right in here as to like narrowing down the results that I want to have and the way we do that then is by adding a wear clause in here so we’ll do uh Dot where and inside of here we can then say that this page and then do in with an underscore one of these Pages all right so if we run that now it’s going to go based off of those pages and so if I were to do let’s just say the about page here those are what’s happening on that about page over that time over that time bucket so if we do one day not going to be that big of a deal right now but of course over time hopefully we’ll see those days come through and it’ll end up working based off of this data so it’s aggregating I think really well and it’s doing so with all the chunks that are available and all that so it’s really really nice and we have the ability to do all sorts of advanced querying in here the other part about this is our wear claw we could do it based off of date time as well so if we were to bring in additional Imports like from date time we can import date time and time Delta and then time zone right and so what we can say something like this where we say start equals to date time. now. time zone. UTC and then we can subtract time Delta and we could do something like hours equals to one and then we could do finish or end being basically the same thing and then we can just go ahead and add in another time Delta in here uh which would allow you to come in and say that the time so event model. time is greater than start comma and then event model. time is less than or equal to finish now that actually filters it down a little bit more but the time bucket is already doing some of that for us so if we were to go into the day and let’s say for instance it’s doing a Time bucket of the day but we actually narrow it down a little bit it’s probably going to give us different results so let’s see here we’ve got day the about page is 7500 but that’s happening within the last hour for today basically if I were to get rid of those and run it again it’s quite a bit more right so it is a the ability for us to filter down if we wanted to this I think is maybe potentially redundant to doing the time bucket because the time bucket already does that really effectively but you will see this from time to time especially when you go into the time bucket Gap fill then you might want to fill in the gaps of a specific interval of time now the thing about this bucket here is you could still do a lot of this stuff in standard uh postgres itself so these SQL queries can get a lot more complex of course um and then what we have here it just so happens that you know hypertable and more specifically time scale has some of these functions built into it that allow for it to happen efficiently so this kind of aggregation is very efficient and then we can take that compiled query let’s go ahead and get rid of the printed results here let’s take that compiled query from the bottom which would be pretty big so I’m going to go ahead and actually start from the top and hopefully grab that whole thing and we will bring it into popsql here and we’ll go ahead and run that new query which let’s make sure that it does finish with page at the end there it looks like it does then we’ll go ahead and run this and now it does basically that exact same query but we can just test it right inside of SQL itself uh which is exactly why we have that compiled query there in the first place um but of course you might not actually run through the SQL stuff very often you might just be running it with python but the underlying SQL is definitely still there and it works essentially the same way as SQL model does here so that’s pretty cool it’s a very straightforward way to do these aggregations and be able to start analyzing our data so all we really need to do is turn this into an API route that will allow for us to do basically this stuff so let’s take a look at how we do that now it’s time to bring in that time bucket querying and aggregating into fast API the way I’m going to do this is I’m going to actually remove our old list view our old events list view to being specifically just aggregations now when we start getting a lot of data it’s really unlikely that I’ll use the list view very often if at all to see individual items I could always bring it back later if I wanted to but realistically I want to see aggregations here this is going to be that lookup so the way this is going to work is by going into our model here we can grab all of the things related to the query I don’t need the compiled query and I will use results in here in just a little bit but the idea is bringing in this query and getting rid of what we had before so I’ll go ahead and paste this in this of course means that I’ll need to import several things which of course I already have some of the Imports on here so I’ll go ahead and bring those in as well as well and then we need the time scale DB import as well so let’s go ahead and do that too okay great so that should give us our new query in here and then the result here I’m going to go ahead and do fetch all and that is actually what I want to return now is just simply those results so that of course means that I need to update my response model so I’m going to leave some of the default arguments in here to start but I do want to change the response model now the response model itself is going to be very uh much based off of what we already saw in here so if we were to run this again we can see that it’s a datetime object the page and then some sort of count in here so that’s what I want to have as well but I want to be a little bit more specific about that before I create the event list schema or the re response model together so that means inside a select I want to add a label to each one of these this label we can go ahead and call it bucket this one is going to just simply going to be page and then this one right here is going to be labeled as count what do you know basically the same thing as what they are but just adding this label on here which is a characteristic of SQL model uh will make it really easy to then have a response model of some kind now let’s remember what’s happening here this query is going to be searching more than one it’s always going to return back a list which means that I also want to bring in from typing we’re going to import list here and we’re going to go ahead and use a list of some kind so back into models.py let’s go ahead and actually create our new schema which is going to be Loosely based off of this event schema not quite the same but count will be the same then we want to have page in here which is a string itself and then we’re going to have a bucket in here which is a datetime object right and so we already have that imported in here because of updated but the idea is this is now going to be our event uh we can call this our event bucket schema something like that and then we’ll go ahead and bring this back into our route we’ll use that as our import now and that’s what we’re going to list back that’s what’s going to be our new response to this request so not a huge difference now one of the things we could think about too is maybe we change this query to being in its own model or its own field here like maybe something like services. that’s not something I’m going to do right now because I really just want to work on what we got here now the other things I’m going to get rid of are this start and finish I don’t necessarily need that I’m just going to go based off of the bucket itself um you totally could do the start and finish in there but using just the bucket will allow for me to do these as query parameters which we’ll do very soon but now that we’ve got this API endpoint let’s just make sure that our server is running uh so we’ll go up here and it looks like we need to maybe save models up pi and there’s our oh we got a little little error there let me just resave that there we go so it looks like everything’s running both in python as well as Docker great so now I’m going to go ahead and make an API call which of course we already have that endpoint in there with list data types I can go ahead and run each one of these I might need to restart the server here or restart the notebook itself um but there’s my results right there right and of course these are off of one day so I want to be able to change this I want to be able to come into my request and do pams and say something like duration and we’ll go ahead and change that to 1 minute right I want that to be a URL parameter that I can change and maybe in here I also want to go ahead and say pages and add in something like slab and let’s see another one maybe contact right and so there we go great so those are the parameters I want to bring in now we can do this fairly straightforwardly inside of fast API which uses a query parameter so the query parameter we can bring in as page and this is going to be an st Str or rather not page but we called it duration and this is going to be a string of some kind which basically will be a default of one day okay so that query we need to bring in to from Fast API we’ll go ahead and import query in here and so having a default in there is nice because then I can use that as my parameter here let’s make sure we put a comma great okay so the next one is going to be our Pages this is going to be a list um but the query itself we’ll go ahead and have a default of none or maybe just a default of an empty list let’s just try with none for now um and we’ll leave it as is okay so this is going to be our default Pages now the pages itself I’m going to go ahead and call these lookup pages and I’ll have some defaults already in here we could always say something along the lines of our default lookup pages and set that up here so we can kind of readjust that as we see fit but we’ll have some default lookup pages in here that will basically be this right here and we’ll go ahead and use those inside of our query here for that lookup great so the idea then is inside of here is if there are Pages coming with the query we would say uh Pages basically and we would then want to check if is instance of a list and pages is greater or the length of pages is greater than than zero else we will go ahead and use this so a nice oneliner for the condition basically if all of these conditions are met it’s going to use those pages otherwise it’s going to use the default ones and then that’s now our lookup and now our API endpoint with that time bucket and all of those aggregations let’s just verify that this is working by going back into our notebook and there we go so we’ve got it at 1 minute here’s that one minute we could also verify this by taking a look at the time that is happening obviously if you had a lot more data it’d be even more clear uh then we can also say something like one week right so we can change it as we see fit and it shows only those items in here if I were to use an empty list here it’s going to then go ahead and use all of them that I have available in my default ones now in my default ones I’m actually missing one which is when I was playing around myself I didn’t actually put a uh leading slash on pricing as well so I did it in two ways so when you actually end up doing this you might want to have some default pages that are in there uh but overall once I actually add that other one and we can see that that one has a lot of data as well now this distribution is probably unlikely I doubt about page would you know be far greater than anything else uh but the idea here or even the contact page being that high right so the analytics here is the data points are actually only because when we created it it just did some pseudo random creation the actual data once you put this into action would look quite a bit different okay so pretty cool now we’ve got a actual API point to get the aggregations from our analytics we have a way to create the data we have a way to aggregate the data both things are what’s critical here and what we will be using once we actually put this thing into a deployed production environment we now want to augment our event model so it’s a a little bit closer to extracting real web data so like for example you want to know what web browser people are using to access these Pages or you want to know how long they spent on those pages now that is more realistic web analytic data now the point of this is to show you how you can add additional fields and what you would need to do to be able to use the time bucket equation to make sure that you can also include those fields and see how those might play out so the idea here then is inside of our event model we’re going to change it now before I make any changes here I’m going to go ahead and run the docker compos down- V to make sure that all of the data has been removed now the reason for this is because I don’t have database migrations in place to make these changes you could have database migrations in place I don’t so we’re going to go ahead and remove some of the fields that are in here by default I’m going to go ahead and get rid of the comments even for them because we no longer need those but I want to add in additional Fields the ones I’m going to add in are these right here we’ve got our user agent otherwise known as like the browser the web browser that’s being used their IP address to identify their machine itself um the refer like you know if it’s coming from Google or from its own website the session ID which you might have in there as well and then how long they spend now of course you could always have more data than just this as well so what we’re not going to do though is change this data we’re just going to ingest this raw data and we will be able to change it later that’s more specific to the user agent so that you can see the specific like operating system that’s being used as we’ll see in just a moment so now that we’ve got this event model I also want to get rid of some of the schemas that are going to be a little bit different or at least change them so the create schema is no longer going to have this description in here it’s just going to have these things the update schema I’m not going to allow updates any longer so I’ll go back into my routing and I will get rid of all of the things related to updating any of these events seeing how to update things was important but now we no longer need them so go ahead and get rid of that as well and of course you could always review the old commits to see all of that old data if for some reason you wanted to go back and see it so at this time we don’t actually have that much different in terms of our data itself so let’s go ahead and bring back Docker I’m going to go ahead and do Docker compose up watch and then I want to make sure that my models everything’s saved and I should see something along the lines of we’ve got a database in here great so these actual Fields themselves maybe you want to change them in the future maybe these aren’t actually the ones you’ll end up using that’s not really the point here the point here is how do we add fields to the data we want to collect and then how do we look at that in terms of our events themselves so this actually means that we need to modify the data we’re going to send so inside of our data here I’m going to go ahead and do a pip install Faker Faker is a python package that allows for us to have some fake data and I’m going to go ahead and grab some of the data that I have which is going to be just this import here I have additional pages in here now I have 10,000 events that I want to bring in Faker can allow for you know fake session IDs in here we can have however many that we want in this case I’m just going to use 20 but if you want to use 200 feel free to do it we still have the same AP in points I added a few refers in here and so we can actually start the process of building out this data so what we are going to do here is very similar to what we have up here where we’re going through a range of events and we’re going to get a random page which we have right there then we’re also going to go ahead and get a random user agent which we can use Faker for so Faker has all of these user agents in here that you can go off of right so just make sure that you have all of that fake stuff implemented then we’re going to go ahead and just create a payload based off of this data so the payload is going to be well simply these key value pairs in here it’s really just a dictionary but I like to looking at it this way uh just to make sure that it’s all working great Okay so we’ve got a duration maybe we do 1 to 300 there’s a lot of things that we could do in that realm but the point here is we now have a new payload that I can send back into my database or into my actual API service with all of that data so I’m going to go ahead and run it and there it goes so we can see all of the different data items that are coming through in here uh we see the user agent and all that okay so while that’s running I’m going to jump into my queries from the SQL model now actually what I want to see is the list data I’m going to go into the API request themselves and we’ll just go ahead and run this within that API request there we go we actually see the same sort of data coming back so if I say something like five minutes we should see a lot of that same data coming back as well now the pages are going to be a little bit different because of my original routing how I had it set up so let’s go ahead and grab these new pages in here as our new default pages so go ahead and copy this and then we’ll go jump into our routing and our new defaults are going to be those right there great now of course you don’t necessarily have to even narrow down the defaults you could remove that Al together which would allow for all of the pages to show up okay so with that in mind now that we’ve got that there we’re going to go ahead and run this once again in that list data types let’s go ahead and run it and there we go so now we have a bunch of different pages in here and we can see the contact and all that stuff okay so now what we want to do is we want to see this a little bit different so going back into the routing we want to add a new field in here and we want to count the number of instances for that field in other words most of this is basically the same so let’s go ahead and do that with let’s say for instance our user agent so if we came in here and we did event. user agent and then we added a label as something like UA like user agent we could do something along these lines where we’re now selecting that data if we go to group that data we totally can which would just be adding in that user agent and then we can order it by that data as well depending on how we see fit our result results now will be slightly different than before so we have to remember the way we are returning our data is going to play in right now so event bucket schema we need to make sure that we are using the label of UA make sure that that’s in there as well so the event schema here we’ll go ahead and bring it in as simply UA this might have a or not UI but UA this might have an optional value in here as well uh which you just set to an empty string maybe uh but we’ll go ahead and save that we’ll save this and then we’ll go back into our list data here and we’ll run it again and now what we see is these things being sort of collected together and right now it’s only showing one count for some of these different user agents so realistically this isn’t that great because it’s not really parsing the data the way I want to because there’s different versions and stuff well if you want to get super granular this is going to be great but we don’t want to get that granular that’s maybe too granular for us so what we want to do is take it a little little further and we’re going to change how we actually return back this data so back into our routing here what we can do is we can import another SQL Alchemy function here so I’ll come in and bring in something called case and we can add in something called a Operating System case so inside of my read events now I can come in here and say something like this where we’ve got our event model user agent and then it basically says hey if it’s one of these things it’s going to set it to something else and then that’s what it’s going to end up being otherwise it’s going to just mark it as other this now I can use as my user agent in here the label I’ll leave as operating system so we’ll go ahead and paste this in here then we will paste it here and here so basically taking place of our original user agent although it’s still the user agent once again I made some changes to the query which will affect the results which means that I need to update the schema itself in which case I’ll go ahead and copy the UA this time I’m going to go ahead and add in the operating system in here same sort of idea still optional let’s go ahead and leave that as is operating system is the label which is why I named it that way we have all these other labels for that same exact reason so there we go we save it let’s go back into our list view here do a quick search and now we’ve got a much better look at what this data could be we’ve got Android at the homepage Android at the about page how many times when all of that and we can keep going through and really see very very robust data so this concept here we could take even further right so we could use something like OS case to unpack different IP addresses so we can see the different parts of the world we can do all sorts of that within the query now I’m not going to go through those Advanced queries in this one but that is absolutely something You’ be able to do at this point let’s go ahead and take a look at how we might do the average duration so I’m going to go ahead and copy the operating system thing here and go ahead and do AVG duration this is going to be optional float and then we’ll go ahead and add in 0.00 or something like that where it’s an average duration value now back into our routing here what we can do to add that average duration is just put it into our select what you do that is funk. AVG then you grab the average of what that value would end up being which is the event model duration and then you set a label to it and it’s average duration and then there we go so now we have enriched this data with the average duration everything else doesn’t need a change because how we’re grouping it together is going to be based off of that operating system and that page and the time not the average duration that wouldn’t actually make any sense to group it by the Aver duration in this place okay so now that we’ve got that let’s go ahead and take a look inside of our list here if I do a quick search here um what I see is that average duration is coming through on each one of these and what we should be able to do is actually modify that to let’s just modify how we were sending this data in the first place by going back into send data here we’re going to change the average duration up by a lot I’ll go ahead and say 50 to uh 5,000 seconds which is definitely a much different look at the durations themselves which should give us a different response back to the actual data that we’re getting depending on our our you know duration parameter here so if I said every 15 minutes for our duration we run that our average durations will hopefully change at least somewhat now of course there would be a bunch of ways on how we can modify this but here’s another average duration that just popped up a lot that’s because of that change that we just did um there was a chance that that wouldn’t have happened but it’s overall really nice that we’re able to completely change how our analytics works and we can do it in a very short amount of time now this is where spending a bunch of time to build out more robust queries might be really useful we now have the foundation in place for a very powerful analytics API where we control all of the data that’s coming in and ignore the data we don’t want then we can also analyze this data with our very own queries now this data itself the actual SQL data that’s coming through the table design probably could be done in postres by itself but the real question here is how efficient or how effective will it be as your data set grows a lot and the answer is it won’t be the nice thing about time scale is you can just add it whenever you need to so if you want to start with just a SQL model you totally can do that now of course the package that we went through won’t necessarily automatically do that just yet maybe at some point it will but right now it doesn’t do that you kind of have to decide this from the get-go in terms of this particular package but in the long run in terms of time scale you can add this at any time and there’s a lot of options for that which is really nice and it gives us a lot of flexibility and the fact that it’s open source means that we can just activate it in our postgress database and We’re Off to the Races but what we want to see now is we want to take this to the next level which is actually deploying it so we can see it in production what it might actually function like on our own systems let’s take a look at how to do that in the next section we’re now going to deploy our application into production using time scale cloud and Railway now time scale Cloud itself will allow for us to use time scale but not worry about running time scale it’s a managed version so all of the performance gains the new releases the bug fixes all of that stuff is going to be in the managed version so we don’t have to worry about it at all now do keep in mind that time scale is based on postgres so it’s still just a postgres database so yeah we could still use the open source version of time scale if we want but it will be a lot more simple if we just go and use time scale Cloud directly so go to this link right here so I get credit for for it and they know that we should make some more videos covering all of these things then we’re going to be deploying our containerized application into Railway now we’ve already seen how to build out the containerized application so this process is going to be fairly straightforward as well so we’ll be able to integrate the two of them and just have it run and we’ll test all of that in this section let’s jump in before we go into production I’m going to go ahead and add in some Kors middleware this is cross origin resource sharing Kors allows us to prevent certain websites from accessing our API this also means certain HTTP methods as well now we’re actually going to open up the floodgates on it mostly because we can actually turn this app into a private networking resource in other words the other apps that would access it need to be inside of that same network that is deployed similar to like you can’t access my version on my computer because it’s in its own private Network that you are not a part of that’s kind of the same idea when we go into production so for that reason I’m going to go ahead and open up my course here so we’ll go ahead and do from fast api. middleware does we’re going to import the cores middleware and then underneath the app here I’m going to go ahead and do app. add middleware and it’s going to be our cores middleware like this and of course you can go into the fast API documentation and get all of the different arguments you might put in here in our casee we’re going to allow all Origins all methods all headers we’re letting everything come through but you know if you were going to expose this to the outside world you might want to lock down the origins to like your actual website domain you might only want to allow get and post methods in here maybe not even delete right so that’s something else that you can think about going forward but this is one part that I wanted to make sure we had before we went into full production so now what I’m going to do is I’m going to push this into GitHub so what we’ve got here is in my terminal I’ve got git status I can actually see all of the files that I’ve changed now I’m actually not going to show you this process mostly because I also have git remote- V I actually have the entire code that you’ll be able to use in the next part we’ll actually use this code directly make a few changes so we can do the deployment directly I won’t do anything else inside of this project at this time other than just adding those cores and that will be on GitHub just a moment now what we’ll do is take it from GitHub to deploy it at this time go ahead and log into GitHub or create an account if you don’t already have one this account that I’m using is really just for these tutorials then you’re going to want to go to cfsh GitHub this will take you to the coding for entrepreneurs profile in which case you’ll go into the repositories here and you’ll look for the analytics API repository of course do a quick search for it if you need to it’s it’s going to be this one right here now the point of this is really to just go ahead and Fork this into your own project here so you go ahead and create that fork and it’s going to bring the code on over now what we need to do is we need to add some additional configuration to our now forked project now that additional configuration has everything to do with deploying fast API so if you actually go to fastapi container.com it’s going to bring you to this in which case will allow you to scroll on down and you can see the code directly that’s going to be used here now this code is just boilerplate code to deploy fast API into Railway that’s it so in our case we really just want this railway. Json because it’s something I’ve worked out to make sure it works really well for you now I’m going to go ahead and copy the contents of this which you can you know select all of it or just press this button right here then I’m going to go back into my repo the one we just forked from the coding for entrepreneurs profile then we’re going to go ahead and hit add file we’re going to create a new file here and I’m going to call this railway. Json and I’m going to paste this on in here now before I commit the changes I will say that it would have been or it will be a good idea to have it on my local project as well I’m just sort of assuming that you are not using git or you haven’t been this whole time maybe you don’t know it yet but of course if you do know it you know what to do from here but the idea here is we need to adjust our Railway settings to ensure that this thing actually gets deployed now what we see is this build command this build command is looking for a Docker file that does not have a period it just says Docker file so we need to use Docker file. web because that’s our actual Docker file path which of course is going to

    be the same for our watch patterns so we actually seen something like these watch patterns already composed igl has watch patterns what do you know now we actually didn’t update our composed yl very well this one probably should say web as well but the point here is we want to actually rebuild this application based off of these patterns here mostly for the SRC and requirements those are the main ones of course but Docker file is another one that’s important as well then we also have this deploy stuff this deploy stuff is looking for that health check do you remember when I said we will have a health check well if we actually look in the main.py code and scroll on down here’s that health check right there notice there’s not a traing slash here but this one is looking for a trailing slash let’s get rid of that TR trailing slash we want to make it the same as what’s in our code so now that we’ve got this I can go ahead and commit these changes and we’ll go ahead and say create railway. Json great our code is now ready for Railway there’s not really much else we need to do to deploy it so let’s go ahead and actually do that going to railway.com feel free to sign up for a free account jump into the dashboard notice that I’m on the hobby account this is a key part of this we’re going to go ahead and jump on over into our account settings and we want to make sure that our account Integrations are connected to GitHub so go through that process if you haven’t what you’ll end up seeing is something similar to this when you go through it and then it’s going to say hey what repositories do you want you could say all of them or you could select the one that you just forked which is what I’m going to do obviously I have two in here but I definitely want to have the one I just forked so I’ll go ahead and save that in here so that now I can go back into Railway and I can deploy this so let’s go ahead and do new now and notice that the analytics API is there it’s ready to go it’s ready for me to deploy it which is really cool it’s a very straightforward process in terms of the deployment but the key here is we want to look at our settings inside of these settings we should be able to scroll on down and we should see the Builder that it says build with a Docker file using buildkit mostly that it says Docker file. web in there if your does not say that that most likely means that your railway. Json in here is done incorrectly which might mean that that isn’t correct that’s kind of the idea beyond that we actually probably won’t need to change anything much notice that it has watch paths in there as well we should not have a start custom start command that’s not necessary one of the things we might need to change is the actual region but I’ll stick with this region for right now and then if you scroll on down notice the health checks in there the timeouts in there all of this stuff is in here things are looking pretty good okay great so what it actually does is it automatically starts deploying this might deploy it might not I actually think it would fail mostly because we don’t actually have our database yet so let’s go ahead and start spinning up our database and let’s combine the two in here assuming that you’ve already signed up for time scale Cloud you’ll log in and you’ll see something like this we’re going to go ahead and create our first database service right now so the idea of course is we’re using postgres so we can just go ahead and hit save and continue now we can select what region we want to use now the region you end up using will likely be close to you physically or close to where the most of your users are so if you’re doing this for somewhere else you’re going to want to put it in that region itself now Us West Oregon is the closest to me physically but it’s also the closest to where I actually have my app being deployed on Railway so if I come in here into my settings I can see the deployment is in California now there is one for organ in here as well so I could always change that too now that deployment is going to fail as well but I’m going to go ahead and leave it in as organ because inside of time scale I have the ability to use organ as well so now I’m going to go ahead and hit save and continue how much data I’m going to be using is practically none because we’re still early days we’re going to use devel velopment here and then we’re going to go ahead and hit save and continue and then we’re going to create this service and I’m just going to call the service name analytics API and then we’re going to go ahead and hit create service now we get a free trial here for 30 days which after that free trial it’s still very affordable in terms of the ability what we can do in here but notice we get a new connection string so I’m going to go ahead and copy this connection string and I’m going to bring it locally first so in myv down here I’m going to go ahead and say CRA DB URL and I’m going to paste that in now the reason I did this is twofold number one if I ever want to test my production URL locally I totally can now this also means that if you skipped using Docker compose this is the route you could do but then you would also use something like this where you actually change the connection string just slightly so realistically you would use something like that you would actually comment out this old one and then use this new one now my case I’m going to leave it as is and I’m going to go ahead and copy the entire new string with it commented out now I’ll go ahead and jump back into Railway into variables here and I can do this raw editor and just paste in those key value pairs right that’s it and then I’m also going to go ahead and update those variables as soon as I do that it’s going to attempt to deploy which I’ll go ahead and let it do now one other thing that’s important to note is how this is being deployed how it’s going to be run so if we go back into our Pro project a long time ago we created this Docker run file which has a run port and a host so this run Port is going to either be set by us or by Railway so we could set this port in here so inside of our variables here we could come in and do a new variable and we can set it to whatever we want I’m going to use 880 which I believe is the default on Railway but I’m going to go ahead and set that port in there as well and then we’ll go ahead and deploy that one as well so one of the nice things about railway is the ability to really quickly change your environment variables and then it will go ahead and build those containers for us and then run them for us very very similar to what we were doing with Docker compose in that sense now we also have this run host here now at some point in the future we might end up using this a little bit differently so at some point if you want a private access one you might need to put your run host the actual host itself inside of a variable as this value right here so you would do something along the lines of run host equ to that in which case I will go ahead and put this into my sample. EMV U you know compos file here just so we have it for later but that might be something if you don’t want to expose this to the outside world okay so after I did all those changes in a matter of minutes it was able to deploy the application itself and we can view the logs in deploy logs this will show us that it is running and notice the port in there says 8080 we also have our build l logs in here we could go through this this is building the container for us it happened really quickly and then it was did the health check which also happen really quickly and then of course it finally deployed so now what we can do is since it is fully deployed we need to of course access this you know API itself so inside of our settings here we go into networking we generate a custom domain now sometimes it might ask you what port you want to use sometimes it might not in this case it just defaulted to the environment variable Port that we have we might even be able to edit that looks like we can right in there which is really nice as well so the one thing that I want to check though is before I go any further is making sure that the deployment is in that region that I wanted and sure enough it is great so going back into our networking this is our API now this is the actual URL that we can use and here’s that hello world and of course if I go into SL API events the actual inpoint itself we should have no events which is very very straightforward so if if you are a casual viewer you will notice now this is a production deployment it’s fully production and it’s also using of course time scale if we go back into our time scale service here and go to the service overview we should see that it has been created right so we’ve got service information in here um we can see all of the things related to it by doing our Explorer in here we can see that there is a hypertable and what do you know there’s that event model all of that is working really well now of course we could always open up popsql and bring in that cloud-based version as well which of course is one of the things it’s built for and you can always test things out there you could also do these all of this stuff locally as well like we discussed okay so what we really need to do though is we need to test this endpoint the actual production endpoint and see if it’s working and how well it’s working let’s take a look let’s go ahead and send some test data to our production endpoint so what I’m going to do here is I’m going to copy the actual URL that I got which of course came directly from Railway it’s this URL here of course you could always have a custom domain as well but I’m going to go ahead and use the one that they gave us and then I’m going to jump into my local notebook here where it says send data to the API and I’m just going to go ahead and change our base URL to that exact value now I don’t actually want the trailing stuff in here I just want to make sure that I’m using htps and then the actual endpoint that it gave us always use htps if you can so now that I’ve got this I should be able to send out a bunch of data this one right here is not actually valid anymore we want to use the new data that we have uh based off of the new schemas that we set up and all the new models so here it is right here I’m going to go ahead and run it it is 10,000 events so in theory it should be able to send all of that data just fine it looks like it’s working fine now if I go into the production inpoint itself refresh an the air there’s some of that data it is now working and it’s working in a way that hopefully you have expected now this of course is now a production endpoint it is fully ready you are ready to deploy this all over the world if you want to and have it in all of your applications and just go wild with analytics but there’s still one more thing that I want to show you and that is how to use this privately now of course we could always sit here and wait for all of this to finish and have all of that data to come through I’ll let you play around with that at this point but let’s actually take a look at how we could deploy the analytics API as a private service so we can still use it this way but just not expose it to the outside world now it probably comes as no surprise that you don’t want to have your analytics API open to the outside world because then you might get events that aren’t accurate they’re not real so of course we need to change that we need to turn our analytics API into a private networking service only so you can have private networking in a lot of cloud services what we’re doing here is we’re really just removing the ability to have public networking which in the case of Railway we can just delete the actual endpoint itself and now there’s no public networking whatsoever but there is private networking and this is something that’s done by default in the case of Railway you have to use IPv6 which means that we need to update how we access this now we could spend a lot of time on Fast API itself to harden the system add security layers to allow only certain connections but as soon as we turn it into a private networking thing it just adds a whole layer of security right there so we don’t have to necessarily add all this additional stuff into our application so what we need to do though is we need to modify our Docker run command here and that’s going to be by changing the host I think I mentioned before that it was the Run host but it’s actually the host in here that we need to change and we need to change it to being this right here so that’s what we’re going to do now is we’re going to do host equals to that as our environment variable and it has to be these two Brack here this is how gunicorn is able to bind to IPv6 so IPv6 is that by default it’s ipv4 which is what this is right here so let’s go ahead and grab this and we’re going to go ahead and bring it into our analytics API as a variable we’re going to go ahead and bring it in here just like this go ahead and update it and deploy it now there’s a chance that this won’t work the reason that it might not work has to do with how this string is here it’s possible that this needed string substitution which we’ll see in just a little bit it’s also possible that it work just fine so once again we will see that in a little bit as well now it’s one thing to make it private and it’s another thing to make it private and still being able to access it so the thing about railway that makes it a little bit simpler is if we go into the settings on our application we can scroll on down to private networking and here is the new API endpoint that we can use inside of our Railway deployments the big question is going to be how do we access that without building out a whole another application I’m going to show you that in a moment so what we see on our API though is that it looks like it’s being deployed and it looks like we’re in good shape let’s look at our logs here more specifically our deploy logs it looks like it’s listening at that actual location so that’s actually really good everything else isn’t arrowing out we do see those print statements that’s a good sign so now we actually want to deploy something that will allow for us to test out these communications so I actually created a tool called Jupiter container.com which will take you to this right here which allows you to have a jupyter notebook in a server right inside of Railway in your own environment so the way we deploy this is by going back into Railway hit create into our project here so the important part of this is you are in the exact same project that you’ve been working on notice that I have a couple deleted in here this one I want to go in and I want to use it right next to the analytics API I do not want to deploy a new project which may happen if you just go to Jupiter container and deploy it directly from there so what we want to do then is come back into our application here we’re going to go ahead and create go into template look for Jupiter that’s with a py and Jupiter container you could probably use jupyter lab as well jupyter container works because I know it works that’s pretty much it um okay so the next thing is we want to add in a password here I’m just going to do abc123 that’s going to be a password this is going to be publicly accessible as we’ll see in a moment I’m going to go ahead and deploy this which might take a moment as well so it’s going to also have a volume on here for persistent storage which is actually kind of nice when it comes to wanting to use things okay so the next thing here is though going to be our variables now what I can do is I can add a variable to my Jupiter container that references this analytics API variable so let’s go ahead and do that I’m going to go ahead and hit new variable here and I’ll go ahead and say um let’s go ahead and call it analytics endpoint something like that and then this one I’m going to use dollar sign two curly brackets and I’ll type out analytics API and these are going to be the objects that we have the op and that’s going to be specifically the rail Ray private domain here so that’s the one that I want to have access to inside of this container which kind of connects them together so I’m going to go ahead and add that and we’ll go ahead and deploy it now these deployments do take a little bit of time because it is building a container and then it’s deploying the image based off of what whatever is going on but one of the main things here is as soon as I create that variable a line is created here showing that they are connected in terms of Railway this is important because the actual Jupiter container itself should have a public inpoint as we see right here so it already has one as far as the template is concerned without us doing anything else so that’s really my goal with this analytics API is to have it so easy that we can deploy it pretty much anywhere we need to as long as we have the necessary configuration across the board which of course would include our time scale DB in there as well so now what we want to do is just wait for this to finish so that we can log in to our service once it’s up and ready which we’ll come back to okay so the Jupiter container finished after a couple minutes let’s go ahead and jump into the actual URL for it there it is I did abc123 to log in of course if you forgot what that value was you can always go into your variable here and just look for this Jupiter password this container is meant to be destroyed almost as soon as you open it up so you can delete it at any time as the point so going back into that jupyter notebook here what we can do is we can jump into the volume and I can create a brand new notebook in here and we’re going to call this you know connect to analytics or something like that I’ll just call it connect to API now I’m going to go ahead and import OS here if you’re not familiar with jupyter notebooks well you probably are now all of these NBS here these are all Jupiter notebooks they’re just running inside of our cursor application so what I can do is I can use os. Environ doget and I should be able to print out the environment variable that we used for our analytics API endpoint so I’m going go ahead and do that and there it is right there notice there’s no htttp on there so that’s an important part of this as well so now what I want to do is import something called httpx httpx is basically like python request but it allows us to call the IP V6 endpoints themselves so our actual inpoint our base URL is going to be equal to http col slash this value right here which I’ll go ahead and set up here like this and just do a little magic here not really magic but you know we’ll go ahead and do some string substitution there’s our base URL let’s jump back into our send data to the API thing in here which I’m going to go ahead and copy this whole thing and we’re going to go ahead and bring it into our Jupiter notebook paste it in like that now I don’t have Faker in here I don’t think so I’m going to go ahead and install it so that installation is going to be a matter of pip install Faker and that should actually install it onto the jupyter notebook itself um and then we should be able to run this now of course the base URL here is now this one so I can go ahead and get rid of these two right here and I also want to use htps X not um using python requests so there’s going to be htbx and I think the API otherwise is basically the same let’s go ahead and give it a quick little run and we get a error in here for that one no service known in here so that’s a little bit of an issue that we will have to address before I do though I want to just take a look at the base URL and what do you know I didn’t do some string substitution so let’s go ahead and make sure we do the string substitution as well one would so that we have the correct base URL now I’m going to go ahead and run this and it still might fail now the reason it still might fail has to do with the port value we’ve got connection refused so when we actually deployed the public endpoint in here when we went through that process it asked for that Port value so this is why I actually mentioned it in the first place is so that we can understand that we do need to know the port value now you could go through the same process as the analytics API endpoint I already know what the port value is off top of my head and it’s 8080 so that’s the one we are going to go ahead and use and we want to use that Port right here this would be the same thing as if we were doing this locally as well and so now we should be able to see the same base URL I’m hoping this actually solves the problem it’s certainly possible that it still won’t solve the problem uh but of course if we look into our deployment for this analytics API we should see that it is running at that specific point this is the endpoint right here so instead of using whatever this is right here right we are using the actual private IP address name basically which is this right here okay or a DNS name rather okay so now that we’ve got that let’s go ahead and see if it starts to work it looks like we’ve got no response of okay so httpx doesn’t do the same thing let’s go ahead and actually just print out the data I’m going to go ahead and come back here and get rid of that and we’ll just go ahead and run this there’s that data coming back and it’s quite literally working in a private networking setting pretty awesome if you ask me so the other part about this of course is to verify this data I’m going to go ahead and stop this we don’t need to add in 10,000 things we can just do 10 for example we could also just G use the git command to grab that data as well so I’m going to go ahead and grab this response similar thing it’s going to be the create endpoint still because it’s the same endpoint to do.get and then we can get that data back and we can see what it looks like by just like that we’ll get rid of some of these comments run that and there you go we now have that data coming through as well now of course if we were to change the params let’s go ahead and change that in the case of the duration I think it was we’ll go ahead and say one day and that’s all all change and now the data is going to be a bit different right and we can also change the pages let’s go ahead and do that I’ll come in here and say pages and we go ahead and put in maybe just the uh let’s go ahead and do just the pricing page and we’ll see what that looks like and now it’s only doing the pricing data in there which is not nearly as many because well it’s going through one day which is you’re grabbing different operating systems over that day right so we saw that as well but the point here is we now have a private analytics API that is well deployed and leveraging a lot of cool technology to wrap up this section we are going to want to unload everything that we just did as in delete all of the deployed resources unless of course you intend to keep them in the case of time scale we’re going to come in here and just grab delete service and then we’re going to go ahead and write out delete go ahead and delete that service that of course will delete all of the data that went with it which of course is no big deal because we did a bunch of fake data we also want to jump into rail way itself and delete the entire project which we can do by coming in here whatever that project is you go into settings you go into danger and then you can remove each one of these items by typing out their names this of course is just to make sure that you clean up all the resources you might not be using in the future so I’m going to go ahead and delete both of these and there we go and then I’ll go ahead and delete the project which I think would also delete those other services but this process has changed a little bit since the last time I did it the idea here is we’ve got all of these things being deleted including the services we were just working on so at this time I won’t be able to access that Jupiter notebook at all any longer or our deployed production API endpoint but you now have the skills to be able to deploy it again and again and again because well the repo is now of course open source on github.com so feel free to go ahead and deploy your own analytics API and if you do change it and make it more robust than than what we have here I would love to check it out let me know hey thanks so much for watching hopefully you got a lot out of this now I will say that data analytics I think is going to get only more valuable as it’s going to be harder and harder to compete so it’s one of those things that I think I’m going to spend a lot more time in what I want to do next is really just kind of visualize the data we built I want to build out a dashboard and I encourage you to attempt to do the same thing after I have that Dash dashboard I really want to see how I can integrate it with an AI agent to see if I can communicate or have some chats about the analytics that’s going on but before I can even make use of chats or a dashboard I probably want to use this somewhere for real so those things are actually better and will help me make decisions either way it’s going to be an interesting technical problem and we’ll probably learn a lot and I hope to show you some of this stuff in the near future so thanks to time scale for partnering with me on this one and thanks again for watching I hope to see you again take care

    By Amjad Izhar
    Contact: amjad.izhar@gmail.com
    https://amjadizhar.blog

  • Android TV Apps: Building for Media and Games

    Android TV Apps: Building for Media and Games

    The provided text is primarily from a book titled “Android TV Apps Development: Building for Media and Games” by Paul Trebilcox-Ruiz. It serves as a guide for developers interested in creating applications specifically for the Android TV platform. The book covers essential topics such as setting up development environments, designing user interfaces optimized for television viewing, and building media playback and game applications using the Android Leanback Support library. Furthermore, it explores advanced features like integrating search functionality, incorporating preferences, utilizing the recommendations row on the home screen, and developing multiplayer experiences using local network connections. Finally, the text touches upon the process of publishing Android TV applications to app stores.

    Android TV Apps Development Study Guide

    Quiz

    1. What are the primary development tools required for Android TV app development, and on which operating systems can they be used?
    2. Explain the significance of adding 10% margins to the edge of your layout when designing Android TV apps. How does the Leanback Support library assist with this?
    3. What is the purpose of the LeanbackPreferenceFragment, and what interface must a class extending it implement to handle preference changes?
    4. Describe the functionality of a CardPresenter in the context of a media application built with the Leanback Support library. What are its key methods?
    5. Explain the roles of VideoDetailsActivity and VideoDetailsFragment in a media application. What are the key listener interfaces that VideoDetailsFragment typically implements?
    6. What is the significance of the SpeechRecognitionCallback interface in Android TV app development, and in what scenario would it be used?
    7. Outline the steps involved in making your application’s content searchable via Android TV’s global search functionality. Mention the key components and files involved.
    8. How does the Nearby Connections API facilitate multiplayer gaming experiences on Android TV? Describe the roles of the host and client devices in this context.
    9. What are KeyEvents and MotionEvents in the context of game development on Android TV? How can a utility class like GameController be used to manage gamepad input?
    10. Briefly explain the purpose of the RecommendationService and BootupReceiver components in enhancing user engagement with an Android TV application.

    Quiz Answer Key

    1. The primary development tool is Android Studio, which requires the Java Runtime Environment (JRE) and Java Development Kit (JDK). It can be used on Windows, Mac OS X, and Linux operating systems.
    2. Adding 10% margins accounts for overscan, where the edges of the television screen might be outside the visible area. The Leanback Support library often handles these layout design guidelines automatically for media playback apps.
    3. LeanbackPreferenceFragment is used to create settings screens in Android TV apps. A class extending it must implement the OnSharedPreferenceChangeListener interface to receive callbacks when preferences are changed.
    4. A CardPresenter is responsible for taking data items (like video objects) and binding them to ImageCardViews for display in a browse or recommendation setting. Key methods include onCreateViewHolder to create the card view and onBindViewHolder to populate it with data.
    5. VideoDetailsActivity serves as a container for the VideoDetailsFragment and sets the layout for the details screen. VideoDetailsFragment displays detailed information about a selected media item and typically implements OnItemViewClickedListener for handling clicks on related items and OnActionClickedListener for handling clicks on action buttons (like “Watch” or “Rent”).
    6. The SpeechRecognitionCallback interface, introduced in Android Marshmallow, allows users to perform voice searches within an application without explicitly granting the RECORD_AUDIO permission. This simplifies the search experience.
    7. Making content globally searchable involves creating a SQLite database to store content information, a ContentProvider to expose this data to other processes, a searchable.xml configuration file to describe the content provider, and declaring the ContentProvider in AndroidManifest.xml. The target Activity for search results also needs an intent-filter for the android.intent.action.SEARCH action.
    8. The Nearby Connections API allows devices on the same local network to communicate easily. In an Android TV game, the TV can act as the host, advertising its presence, and mobile phones or tablets can act as clients, discovering the host and exchanging data for second-screen experiences or hidden information.
    9. KeyEvents represent actions related to physical button presses on a gamepad controller, while MotionEvents represent analog inputs from joysticks or triggers. A GameController utility class can track the state of buttons (pressed or not) and the position of joysticks to provide a consistent way to access gamepad input across the application.
    10. RecommendationService periodically fetches and displays content recommendations on the Android TV home screen, encouraging users to engage with the app. BootupReceiver is a BroadcastReceiver that listens for the BOOT_COMPLETED system event and schedules the RecommendationService to start after the device boots up, ensuring recommendations are available.

    Essay Format Questions

    1. Discuss the key design considerations that differentiate Android TV app development from mobile app development for phones and tablets. Focus on user interaction from a distance, navigation with a D-pad controller, and color palette choices.
    2. Explain the architecture and workflow of building a media application on Android TV using the Leanback Support library. Describe the roles of key components like BrowseFragment, CardPresenter, ArrayObjectAdapter, and DetailsFragment.
    3. Describe the process of integrating global search functionality into an Android TV application. Detail the purpose and interaction of the SQLite database, ContentProvider, and the searchable.xml configuration file.
    4. Discuss the challenges and opportunities of developing multiplayer games for Android TV using techniques like local area network communication with the Nearby Connections API and handling gamepad input.
    5. Explain the strategies for enhancing user engagement with an Android TV application beyond basic functionality. Focus on features like content recommendations using RecommendationService and enabling voice search.

    Glossary of Key Terms

    • Android Studio: The integrated development environment (IDE) officially supported by Google for Android development.
    • Android TV OS: The operating system designed by Google for smart TVs and digital media players.
    • Leanback Support Library: A collection of Android support libraries specifically designed to help developers build user interfaces for TV devices.
    • BrowseFragment: A key component of the Leanback Support Library used to display categorized rows of media items.
    • CardPresenter: A class in the Leanback Support Library responsible for taking data and binding it to a visual card representation (e.g., ImageCardView).
    • ArrayObjectAdapter: An adapter class used with Leanback UI components to provide a list of data items for display.
    • DetailsFragment: A Leanback Support Library fragment used to display detailed information about a selected media item, including actions.
    • Presenter: In the context of the Leanback Support Library, an abstract class that defines how data should be displayed in a ViewHolder.
    • ViewHolder: A pattern used to efficiently update views in a RecyclerView or Leanback list row by holding references to the view components.
    • Overscan: The area around the edges of a traditional television picture that may not be visible to the viewer. Android TV development recommends accounting for this with layout margins.
    • D-pad Controller: The directional pad commonly found on TV remote controls and gamepads, used for navigation on Android TV.
    • Digital Rights Management (DRM): Technologies used to protect copyrighted digital content.
    • ExoPlayer: An open-source media player library for Android that provides more features than the standard MediaPlayer class.
    • AndroidManifest.xml: The manifest file that describes the essential information about an Android app to the Android system.
    • Intent-filter: A component in the AndroidManifest.xml that specifies the types of intents that an activity, service, or broadcast receiver can respond to.
    • ContentProvider: An Android component that manages access to a structured set of data. They encapsulate the data and provide mechanisms for defining security.
    • SQLite: A lightweight, disk-based, relational database management system.
    • Global Search: The system-wide search functionality in Android TV that allows users to search across different installed applications.
    • Searchable.xml: A configuration file that describes how an application’s data can be searched by the Android system.
    • Nearby Connections API: A Google Play services API that allows devices on the same local area network to discover and communicate with each other.
    • GoogleApiClient: An entry point to the Google Play services APIs.
    • ConnectionCallbacks: An interface that provides callbacks when a connection to Google Play services is established or suspended.
    • OnConnectionFailedListener: An interface that provides a callback when a connection to Google Play services fails.
    • ConnectionRequestListener: An interface used with the Nearby Connections API to handle incoming connection requests.
    • MessageListener: An interface used with the Nearby Connections API to receive messages from connected devices.
    • EndpointDiscoveryListener: An interface used with the Nearby Connections API to receive notifications when nearby devices (endpoints) are discovered or disappear.
    • KeyEvent: An object that represents a key press or release event.
    • MotionEvent: An object that represents a motion event, such as touch screen interactions or joystick movements.
    • RecommendationService: A service that runs in the background and provides content recommendations to be displayed on the Android TV home screen.
    • BootupReceiver: A BroadcastReceiver that listens for the system’s boot complete event and can be used to start services like RecommendationService after the device restarts.
    • IntentService: A base class for services that handle asynchronous requests (expressed as Intents) on a worker thread.

    Briefing Document: Android TV Apps Development – Building for Media and Games

    Source: Excerpts from “0413-Android TV Apps Development – archive done.pdf” by Paul Trebilcox-Ruiz (Copyright © 2016)

    Overview: This briefing document summarizes key themes and important concepts from Paul Trebilcox-Ruiz’s book, “Android TV Apps Development: Building for Media and Games.” The book guides developers through creating applications for the Android TV platform, covering setup, UI design considerations for large screens, building media playback apps, enriching apps with search and recommendations, and developing games. It emphasizes the use of Android Studio and the Android Leanback Support Library.

    Main Themes and Important Ideas:

    1. Setting Up the Development Environment:

    • Android TV development utilizes the same tools as standard Android development, compatible with Windows, Mac OS X, and Linux.
    • Android Studio is the recommended Integrated Development Environment (IDE) and requires the Java Runtime Environment (JRE) and Java Development Kit (JDK).
    • The Android SDK, including platform tools and APIs (at least Android 5.0 Lollipop at the time of writing), needs to be installed via Android Studio.
    • Creating a new Android TV project in Android Studio involves selecting the TV form factor during project configuration.
    • The base Android TV template provides a starting point, although some initial code might contain deprecated components that can be ignored initially.
    • “One of the nice things about developing for Android is that the development tools can be used on most modern computer platforms, and Android TV development is no different.”

    2. Planning and Designing for the Android TV Experience:

    • Developing for TV requires different considerations than for handheld devices due to the “10-foot experience” where users interact from a distance.
    • Overscan: It’s crucial to account for overscan by adding approximately 10% margins to the edges of layouts to ensure content isn’t clipped on all TVs. The Leanback Support Library often handles this for media apps.
    • Coloration: Televisions can display colors inconsistently. Avoid bright whites over large areas and test dark or highly saturated colors on various TVs. Google recommends using colors two to three levels darker than mobile and suggests the 700-900 range from their color palette.
    • Typography: Specific font families (Roboto Condensed and Roboto Regular) and sizes (specified in sp for density independence) are recommended for different UI elements (cards, browse screens, detail screens). The Leanback Support Library includes styles to manage this.
    • Controller Support: Applications must be navigable using the basic Android TV D-pad controller. For proprietary media players, D-pad compatibility needs to be ensured.
    • Media Player Choice: While the standard MediaPlayer class is available, Google’s open-source ExoPlayer is highlighted as an excellent alternative with more advanced features.
    • “While you may be familiar with Android development for phones and tablets, there are many things you need to consider when creating content for the TV, depending on whether you are making a game, utility, or media application.”

    3. Building a Media Playback Application:

    • This involves creating Activities (e.g., MainActivity, VideoDetailsActivity, PlayerActivity) and Fragments (e.g., MainFragment, VideoDetailsFragment, PlayerControlsFragment).
    • The Leanback Support Library is fundamental, providing classes like BrowseFragment for displaying categorized content rows.
    • Data Presentation: Using ArrayObjectAdapter and ListRow to display lists of media items with headers. Presenter classes (like CardPresenter) are used to define how individual items are displayed (e.g., using ImageCardView).
    • Fetching Data: Demonstrates loading data from a local JSON file (videos.json) using utility classes and libraries like Gson for JSON parsing and Picasso for image loading.
    • Video Details Screen: Utilizing DetailsFragment to show detailed information about selected media, including actions (e.g., “Watch,” “Rent,” “Preview”) implemented using Action objects and SparseArrayObjectAdapter.
    • Media Player Implementation: Using VideoView for video playback and creating a custom PlayerControlsFragment with playback controls (play/pause, skip, rewind, etc.) built using PlaybackControlsRow. An interface (PlayerControlsListener) is used for communication between the fragment and the PlayerActivity.
    • “BrowseFragment will allow you to display rows of items representing the content of your app, preferences, and a search option.”

    4. Enriching Media Apps with Search and Recommendations:

    • In-App Search: Implementing a SearchFragment and a corresponding Activity (MediaSearchActivity). Using SpeechRecognitionCallback to handle voice search without explicit audio recording permissions.
    • Local Search Implementation: Filtering a local data source based on a user’s query.
    • Settings Screen: Using LeanbackPreferenceFragment to create a settings interface. Custom Presenter classes (PreferenceCardPresenter) can be used to display preference options as cards.
    • Recommendations: Implementing a RecommendationService that uses NotificationManager and NotificationCompat.Builder to display content recommendations on the Android TV home screen. TaskStackBuilder is used to create the appropriate back stack when a recommendation is clicked. A BootupReceiver and AlarmManager are used to schedule periodic recommendation updates.
    • Global Search Integration: Creating a SQLite database (VideoDatabaseHandler) to store content information and a Content Provider (VideoContentProvider) to expose this data to the Android TV system for global search. Configuring searchable.xml and the AndroidManifest.xml to declare the content provider and enable search functionality. The VideoDetailsActivity is configured to handle the android.intent.action.SEARCH intent.
    • “Content providers are Android’s way of making data from one process available in another.”

    5. Android TV Platform for Game Development:

    • Android TV is a fully functioning Android OS, making it relatively straightforward to migrate Android games.
    • Focuses on Android development tools for games, acknowledging that other game engines also work.
    • Gamepad Controller Input: Demonstrates how to detect and handle gamepad button presses (KeyEvent) and analog stick movements (MotionEvent). A utility class (GameController) is created to manage the state of the controller. dispatchKeyEvent and dispatchGenericMotionEvent in the main Activity are used to intercept and process input events.
    • Visual Instructions: Recommends displaying visual instructions for using the controller, referencing Google’s Android TV gamepad template.
    • Local Area Network (LAN) Integration: Introduces the Nearby Connections API as a way to create second-screen experiences where mobile devices can interact with a game running on Android TV (acting as a host).
    • Nearby Connections API Implementation: Requires adding the play-services dependency, requesting network permissions, and defining a service ID in the AndroidManifest.xml. Demonstrates how to use GoogleApiClient to connect to the Nearby Connections API, advertise the TV app over the LAN, discover nearby devices (mobile app), and send and receive messages between them using ConnectionRequestListener, MessageListener, and EndpointDiscoveryListener.
    • “Thankfully, since Android TV is a fully functioning Android OS, it doesn’t take much to migrate your games over to the new platform.”

    Key Libraries and Components Emphasized:

    • Android Studio: The primary development IDE.
    • Android SDK: Provides the necessary tools and APIs.
    • Java Runtime Environment (JRE) and Java Development Kit (JDK): Required by Android Studio.
    • Android Leanback Support Library: Essential for building TV-optimized UIs, providing components like BrowseFragment, DetailsFragment, PlaybackControlsRow, ImageCardView, ArrayObjectAdapter, and ListRowPresenter.
    • Gson: For parsing JSON data.
    • Picasso: For loading and caching images.
    • RecyclerView: For displaying efficient lists and grids (used within Leanback components).
    • SQLite: For local data storage (used for global search integration).
    • ContentProvider: For securely sharing data between applications (used for exposing search data).
    • Nearby Connections API (part of Google Play Services): For enabling communication between devices on the same local network.

    Target Audience: Android developers looking to build applications and games for the Android TV platform. The book assumes some familiarity with basic Android development concepts.

    This briefing document provides a high-level overview of the key topics covered in the provided excerpts. The book delves into the code-level implementation details for each of these areas.

    Android TV App Development: Key Considerations

    Frequently Asked Questions: Android TV App Development

    • What are the primary focuses when developing Android TV apps according to this material? This material focuses on building Android TV applications for two main categories: media consumption and games. It guides developers through the specifics of creating user interfaces suitable for television viewing, handling remote controllers, integrating media playback, and adapting game development principles for the Android TV platform.
    • What are the key considerations for UI/UX design when developing for Android TV compared to mobile devices? Developing for Android TV requires considering that users will be interacting with the app from a distance using a remote control. Key considerations include: larger font sizes and text styling optimized for TV screens, using a density-independent sizing quantifier (sp) for text, accounting for overscan by adding margins to layouts, choosing color palettes that display well on various television types (avoiding pure white and checking dark/saturated colors), and designing navigation that is easily manageable with a D-pad controller. The Leanback Support library is highlighted as a tool that assists with these design considerations.
    • How does the Leanback Support Library aid in Android TV app development? The Leanback Support Library is a crucial component for Android TV development. It provides pre-built UI components specifically designed for the TV experience, such as BrowseFragment for displaying categorized rows of content, DetailsFragment for displaying detailed information about media items, PlaybackControlsRow for creating media playback controls, and classes for handling card-based layouts. It also incorporates design guidelines for large screens and remote control navigation, simplifying the development process for media and other TV-centric applications.
    • What are the recommended steps for building a media playback application for Android TV based on this content? The recommended steps include: setting up an Android Studio project and including the Leanback Support library dependency; building a BrowseFragment to display media content in rows with categories, often by parsing a JSON data source; creating a CardPresenter to define how media items are displayed as cards; implementing a VideoDetailsActivity and VideoDetailsFragment to show detailed information and actions (like “Watch”) for selected media; building a PlayerActivity with a VideoView for media playback and a PlayerControlsFragment using PlaybackControlsRow for user controls; and potentially integrating the ExoPlayer for advanced media playback features.
    • How can Android TV apps incorporate search functionality? Android TV apps can incorporate search functionality in two primary ways: in-app search and global search. In-app search can be implemented using the SearchFragment from the Leanback Support Library, allowing users to search within the app’s content. Integrating with Android TV’s global search requires creating a SQLite database to store searchable content information, implementing a ContentProvider to expose this data to the system, and declaring the content provider and a searchable configuration in the AndroidManifest.xml. Activities that display search results need to handle the ACTION_SEARCH intent.
    • What considerations are important for game development on Android TV? Migrating games to Android TV involves adapting to the platform’s input methods, primarily gamepads. Developers need to handle KeyEvents for button presses and MotionEvents for analog stick inputs. It’s crucial to provide clear visual instructions on how to use the controller within the game. While the core Android OS is the same, the interaction paradigm shifts from touchscreens to remote controls and gamepads. Popular game engines are also noted to work with Android TV.
    • How can Android TV applications leverage local area networks for enhanced experiences, particularly in games? Android TV applications can use the Nearby Connections API to enable communication between devices on the same local network. This is particularly useful for creating second-screen experiences in games, where a TV acts as the host and mobile devices as clients, allowing for private information or controls on the second screen. Implementing this involves adding the Play Services dependency, requesting network permissions, defining a service ID, using GoogleApiClient to connect, advertising the service on the host device, and discovering and connecting to the service on client devices, as well as handling message sending and receiving.
    • What are some advanced features that can be integrated into Android TV apps, as highlighted in this material? Advanced features discussed include: implementing in-app search and integration with global search; adding settings screens using LeanbackPreferenceFragment to allow users to customize the app; providing content recommendations using RecommendationService to surface content on the Android TV home screen as notifications; and utilizing the Nearby Connections API for local network interactions, especially for second-screen gaming experiences.

    Developing Android TV Applications

    Developing Android TV apps involves creating applications specifically designed for the Android TV platform, which aims to bring interactive experiences to television sets. This platform, introduced by Google in 2014, is optimized for television viewing and can be found in smart TVs or accessed via set-top boxes. Android TV is built upon the Android operating system, allowing developers to leverage their existing Android development skills and familiar components like activities, fragments, and adapters. The Leanback Support library provides additional components tailored for the TV interface.

    To begin developing for Android TV, you’ll need a modern computer with Windows, Mac OS X, or Linux and the Android Studio development environment, which requires the Java Runtime Environment (JRE) and Java Development Kit (JDK). Creating a new Android TV project in Android Studio involves selecting the TV form factor and a minimum SDK of API 21 (Lollipop) or later, as Android TV was introduced with Lollipop. You can choose an empty project or a default Android TV activity to start. Running your app can be done using an emulator or on a physical Android TV device like the Nexus Player or NVIDIA SHIELD.

    A crucial aspect of Android TV app development is considering the user experience from a distance. Google recommends adhering to three main ideas: casual consumption, providing a cinematic experience, and keeping things simple. This means designing apps that allow users to quickly achieve their goals, utilizing audio and visual cues, limiting the number of screens, and ensuring easy navigation with a D-pad controller. Layouts should be designed for landscape mode with sufficient margins to account for overscan. Color choices should be carefully considered due to variations in television displays, and text should be large and easy to read from a distance.

    Android TV offers several features to enhance user engagement:

    • Launcher Icon: A correctly sized (320px x 180px) and styled launcher icon that includes the app name is essential for users to find your application in the list of installed apps. Games require the isGame=”true” property in the application node of the AndroidManifest.xml to be placed in the games row.
    • Recommendations Row: This row on the home screen provides an opportunity to suggest continuation, related, or new content to users using a card format. Implementing a recommendation service involves creating notification cards from a background service and pushing them to the home screen.
    • Global Search: By making your application searchable, users can find your content through the Android TV global search by voice or text input. This involves creating a SQLite database and a ContentProvider to expose your app’s data.

    The book focuses on building media apps using the Leanback Support library, which provides components like BrowseFragment for displaying rows of content and DetailsFragment for presenting detailed information. It walks through creating a basic media playback application, including handling video playback with VideoView and displaying controls using PlaybackOverlayFragment.

    For game development, Android TV offers similar development tools to mobile but requires consideration for landscape orientation and potential multiplayer experiences using second screens. Supporting gamepad controllers involves handling digital and analog inputs. The Nearby Connections API facilitates communication between devices on the same local area network for second-screen experiences. Google Play Game Services provides APIs for achievements, leaderboards, and saved games.

    Publishing Android TV apps to the Google Play Store requires meeting specific guidelines to ensure proper layout and controls for television users. This includes declaring a CATEGORY_LEANBACK_LAUNCHER intent filter, providing a 320px x 180px banner icon, and ensuring compatibility with Android TV hardware by not requiring unsupported features like a touchscreen or camera. Apps are also expected to respond correctly to D-pad or game controllers and ideally support global search and recommendations. Distribution is also possible through the Amazon App Store for Fire TVs.

    Android TV Game Development

    Discussing game development for Android TV involves understanding how to adapt existing Android games or create new ones specifically for the television platform. While the core Android development principles remain similar to mobile, there are specific considerations for the TV environment.

    One key difference between Android TV and mobile game development is the orientation: Android TV games should primarily, if not exclusively, work in landscape mode. Unlike phones and tablets which can switch between portrait and landscape, televisions are almost always in landscape orientation, so your game’s design and layout must accommodate this.

    When setting up your game project, you’ll need to make some adjustments to the AndroidManifest.xml file. To have your game appear in the games row on the Android TV home screen, you must declare your application as a game by adding the android:isGame=”true” property within the <application> node. If your game supports the gamepad controller, you should also declare the <uses-feature android:name=”android.hardware.gamepad” android:required=”false” /> to indicate this support, but setting required to false ensures your app remains installable on Android TV devices even without a gamepad.

    Handling gamepad controller input is crucial for many Android TV games. Gamepad controllers provide both digital inputs (buttons with pressed/unpressed states) and analog inputs (joysticks or triggers providing values within a range). You can read these inputs through KeyEvent (for button presses) and MotionEvent (for analog inputs). The source mentions creating a GameController.java utility class to store and manage the state of these inputs and provides methods to handle KeyEvent and MotionEvent events. In your game’s Activity or View, you would override methods like dispatchKeyEvent and dispatchGenericMotionEvent to forward these events to your GameController and then update your game logic accordingly.

    There are several controller best practices to follow for a good user experience:

    • Inform users in the Google Play Store description if a controller is necessary.
    • Adhere to user expectations for button functions (e.g., A for Accept, B for Cancel).
    • Verify controller hardware requirements and have a backup plan if certain hardware like a gyroscope or triggers are missing on a user’s controller.
    • For multiplayer games, ensure your app handles multiple controllers by detecting device IDs.
    • When a controller disconnects, pause the game and inform the user.
    • If possible, display visual instructions for using the controller, and Google provides an Android TV gamepad template for this purpose.

    For local multiplayer games requiring secrecy between players, you can implement second screen experiences using the Local Area Network (LAN). Google’s Nearby Connections API facilitates communication between devices on the same network. In this model, the Android TV often acts as the central host, and players can use their mobile phones or tablets as private second screens to perform actions. Setting this up involves creating separate modules for the TV and mobile apps, including the Play Services library, requesting necessary network permissions, and defining a service ID for the apps to identify each other on the network. The TV module would advertise its availability over the LAN and respond to connection requests, while the mobile modules would discover the host and connect. Once connected, devices can send and receive messages to facilitate gameplay.

    Google Play Game Services offers a suite of APIs and tools specifically for game developers to enhance their Android TV games. Some key features include:

    • Achievements: Reward players for enjoying your game, fostering competition. At least five achievements are required for publishing.
    • Leaderboards: Allow players to compare their scores with others.
    • Saved Games: Enable players to save their progress online and resume on different devices.
    • Multiplayer: Support online multiplayer for real-time and turn-based games.
    • Quests and Events: Engage players with time-bound challenges and analyze player actions through in-game events.

    When publishing your Android TV game, remember to declare it as a game in the manifest. If your game supports a controller, ensure it has contingencies for buttons like Start, Select, and Menu, and provide a generic gamepad controller graphic for instructions. It’s also important to ensure your application provides a clear way for users to exit the game and return to the home screen. For networking aspects, verify that your code handles both Wi-Fi and Ethernet connections, as Android TV devices can support both. Like other Android TV apps, games undergo an approval process to ensure proper layout and control functionality on the TV platform.

    Android TV Media App Development

    Let’s delve into the discussion of Media Applications for Android TV, drawing on the information from the provided source, “Android TV Apps Development: Building for Media and Games,” and our previous conversation.

    Media applications are identified as a very common type of application developed for televisions. Recognizing this, Google created the Leanback Support library, which offers a collection of pre-built components specifically designed to facilitate the creation of media apps that adhere to Android TV’s design guidelines. This library is crucial for developers looking to build effective and user-friendly media experiences on the platform.

    The source highlights several key components from the Leanback Support library that are fundamental to building media applications:

    • BrowseFragment: This class is presented as a core part of an Android TV media app. While appearing as a single fragment to the user, it’s actually composed of two underlying fragments:
    • RowsFragment: Responsible for displaying vertical lists of customized cards that represent your media content. Each row is typically associated with a category.
    • HeadersFragment: This forms the teal “fast lane” panel often seen on the left side of the screen, populated with headers corresponding to the categories displayed in the RowsFragment. The BrowseFragment uses an ObjectAdapter to manage the list of content (rows), and each item within a row is associated with a Presenter object. The Presenter dictates how each media item will visually appear in the UI, often as a card with an image and title. The CardPresenter is a concrete implementation used for this purpose.
    • DetailsFragment: While the BrowseFragment offers a quick overview of available media, the DetailsFragment is designed to focus on a single item. This screen serves to provide more in-depth information about the content, present various actions that can be taken (e.g., “Watch,” “Add to Favorites”), and potentially display related media. The DetailsFragment often utilizes a DetailsOverviewRowPresenter (though the source recommends the FullWidthDetailsOverviewRowPresenter as the former is deprecated) to display a logo, a row of actions, and a customizable detail description. It also works with ListRowPresenter to display related media.
    • PlaybackOverlayFragment: For applications that involve playing media, the PlaybackOverlayFragment is essential for displaying media controls to the user. This fragment provides a user interface for actions like play/pause, fast forward, rewind, and potentially more advanced controls. It works in conjunction with PlaybackControlsRow to present these actions.

    The source emphasizes that when designing media applications for Android TV, developers must keep in mind the unique context of television viewing. The design guidelines discussed in our previous turn are particularly relevant:

    • Casual Consumption: Media apps should be designed to get users to the content they want to enjoy as quickly as possible, with minimal interaction required.
    • Cinematic Experience: Utilizing audio and visual cues, along with animations and transitions (while avoiding overwhelming the user), can enhance the immersive quality of the media experience.
    • Keep It Simple: Navigation should be straightforward using the D-pad controller, minimizing the number of screens and avoiding text entry where possible. The list of rows pattern seen on the home screen is a recommended UI approach.

    To further engage users with media applications, Android TV offers several key features that developers should consider integrating:

    • Launcher Icon: A distinct and correctly sized (320px x 180px) launcher icon that includes the app’s name is crucial for users to easily find and launch the application from the home screen.
    • Recommendations Row: This prime location on the Android TV home screen allows media apps to suggest content to users. This can include continuation of previously watched media, related content based on viewing history, or highlighting new and featured items. Implementing a RecommendationService is key to populating this row with engaging content presented in a card format.
    • Global Search: By making the application’s media library searchable through Android TV’s global search functionality, users can easily find specific movies, shows, or other content using voice or text input, regardless of which app it resides in. This requires creating a SQLite database to store content information and a ContentProvider to expose this data to the system. A searchable.xml configuration file and an intent filter in the media details activity are also necessary.
    • Now Playing Card: For media that can continue playback in the background (like audio), providing a “Now Playing” card in the recommendations row allows users to quickly return to the app and control playback.
    • Live Channels: For apps offering linear or streaming content, integration with the Android TV Live Channels application via the TV Input Framework allows users to browse your content alongside traditional broadcast channels.

    The source provides a practical guide to building a basic media application step-by-step, covering project setup, manifest configuration, implementing a BrowseFragment to display media items, creating a VideoDetailsActivity with a DetailsFragment to show more information and actions, and finally, implementing basic video playback using a PlayerActivity and the PlaybackOverlayFragment for controls.

    Furthermore, the book delves into enriching media applications with features like in-app searching using a SearchOrbView and SearchFragment, implementing a settings or preference screen using LeanbackPreferenceFragment, leveraging the recommendations row, and integrating with Android TV’s global search functionality.

    Finally, when it comes to publishing media applications, it’s essential to adhere to the Android TV app checklist, ensuring that the UI is designed for landscape orientation and large screens, navigation is seamless with a D-pad, and that features like search and recommendations are properly implemented to enhance content discovery.

    In summary, developing media applications for Android TV leverages the Android framework and the specialized Leanback Support library to create engaging entertainment experiences optimized for the television screen. Careful consideration of the user experience from a distance and integration with Android TV’s unique features are key to building successful media apps on this platform.

    Android TV Leanback Support Library: Development Overview

    The Leanback Support library is a crucial set of tools provided by Google to facilitate the development of applications specifically for the Android TV platform. This library is designed to help developers create user interfaces and experiences that are optimized for the television screen and remote-based navigation.

    Here are the key aspects of the Leanback Support library, drawing from the sources and our conversation:

    • Purpose and Benefits: The primary goal of the Leanback Support library is to simplify the development of engaging entertainment applications for Android TV. It does this by:
    • Demystifying new Android TV APIs.
    • Providing pre-built and optimized UI components that adhere to Android TV’s design guidelines. This helps ensure a consistent and familiar user experience across different Android TV apps.
    • Offering the necessary tools for building applications that run smoothly on the Android TV platform.
    • Helping developers understand the specific vocabulary and concepts relevant to Android TV development.
    • Providing practical code examples to guide developers in implementing various features.
    • Offering insights into design considerations that are unique to the television environment, leading to more enjoyable user experiences.
    • Taking layout design guidelines into account, such as overscan, particularly for media playback applications.
    • Key UI Components: The Leanback Support library includes several essential UI building blocks for Android TV applications, especially media apps:
    • BrowseFragment: This is a core component for displaying categorized rows of media content. It essentially comprises a RowsFragment for the content cards and a HeadersFragment for the navigation sidebar (the “fast lane”). It utilizes ObjectAdapter and Presenter classes (like CardPresenter) to manage and display media items.
    • DetailsFragment: Used to present detailed information about a specific media item, along with available actions such as watching or adding to favorites. It often employs DetailsOverviewRowPresenter (though FullWidthDetailsOverviewRowPresenter is recommended) and ListRowPresenter to display details and related content.
    • PlaybackOverlayFragment: Essential for media playback applications, this fragment provides a user interface for controlling the playback of media content. It works with classes like PlaybackControlsRow and various Action classes (e.g., PlayPauseAction, FastForwardAction).
    • SearchFragment and SearchOrbView: These components enable the implementation of in-app search functionality, allowing users to find specific content within the application.
    • LeanbackPreferenceFragment: A specialized fragment for creating settings or preference screens that adhere to the visual style and navigation patterns of Android TV.
    • GuidedStepFragment: Provides a way to guide users through a series of decisions using a structured interface with a guidance view and a list of selectable items.
    • Support for Android TV Features: The Leanback Support library also provides mechanisms to integrate with key Android TV platform features:
    • Recommendations: The library helps in building services (RecommendationService) that can push content suggestions to the Android TV home screen’s recommendations row, enhancing user engagement.
    • Global Search: While the library doesn’t directly implement global search, the UI components it provides can be used to display search results effectively. Integrating with global search requires using Android’s SearchManager and ContentProvider as discussed in the sources.
    • Design Considerations: Apps built with the Leanback Support library inherently encourage adherence to Android TV’s design principles, such as casual consumption, cinematic experience, and simplicity in navigation. The library’s components are designed to be easily navigable using a D-pad controller, which is the primary input method for most Android TV devices.

    In the context of our previous discussions:

    • For media applications, the Leanback Support library is indispensable, providing the foundational UI structures and controls needed for browsing, detail views, and media playback.
    • While our game development discussion focused more on gamepad input and networking, the Leanback Support library also plays a role in the UI of Android TV games, particularly for menus, settings, and potentially displaying game-related information in a TV-friendly manner. Components like GuidedStepFragment could be useful in game tutorials or settings screens.

    In summary, the Leanback Support library is the cornerstone for developing high-quality Android TV applications, especially in the realm of media and entertainment. It offers a rich set of UI components and assists developers in adhering to platform-specific design guidelines and integrating with key Android TV features, ultimately leading to better and more consistent user experiences.

    Android TV App Publishing Essentials

    Let’s discuss App Publishing for Android TV, drawing on the information from the sources and our conversation history [Discuss Leanback Library].

    The Android TV App Publishing Process and Checklist

    Before publishing your Android TV application, it’s crucial to ensure it meets Google’s guidelines for approval. This approval process isn’t for censorship but to verify that your app’s layouts and controls function correctly for Android TV users. Google provides an Android TV App Checklist that you should validate before uploading your APK to the Play Store.

    Key items on this checklist, according to the sources, include:

    • Support for the Android TV OS:
    • You must provide an Android TV entry point by declaring a CATEGORY_LEANBACK_LAUNCHER intent filter in an activity node of your manifest. Without this, your app won’t appear in the application rows on the home screen.
    • Associate a banner icon (320px by 180px) with this activity, which will be displayed in the application row. Any text on the banner needs to be localized.
    • Ensure your manifest doesn’t declare any required hardware features not supported by Android TV, such as camera, touchscreen, and various hardware sensors. If these are marked as required, your app won’t be discoverable by Android TV devices.
    • UI Design:
    • Your app must provide layout resources that work in landscape orientation. Android TV primarily operates in landscape mode.
    • Ensure all text and controls are large enough to be visible from an average viewing distance (around ten feet) and that bitmaps and icons are high resolution.
    • Your layouts should handle overscan, and your application’s color scheme should work well on televisions. As we discussed with the Leanback Library, its components are designed with these considerations in mind.
    • If your app uses advertisements, it’s recommended to use full-screen, dismissible video ads that last no longer than 30 seconds. Avoid ads that rely on sending intents to web pages, as Android TV doesn’t have a built-in browser, and your app might crash if a browser isn’t installed.
    • Your app must respond correctly to the D-pad or game controller for navigation. The Leanback Support library provides classes that handle this. Custom classes should also be designed to respond appropriately.
    • Searching and Discovery:
    • It’s highly recommended that global search and recommendations are implemented and working in your application. Users should be taken directly to the content they are interested in when found through search or recommendations. We discussed implementing these features in detail earlier.
    • Games:
    • If your app is a game, you need to declare it as a game (android:isGame=”true”) in the application node of the manifest to have it appear in the games row on the home screen.
    • Update your manifest to reflect support for the game controller if applicable.
    • Ensure your game has button contingencies for Start, Select, and Menu buttons, as not all controllers have these.
    • Provide a generic gamepad controller graphic to inform users about the controls.
    • Your application needs controls for easily exiting the game to return to the home screen.
    • For networking in games, ensure your code verifies network connectivity via both WiFi and Ethernet, as Android TV can support Ethernet connections.

    Distributing Your Application

    Once your app is complete and meets the guidelines, you can distribute it through major outlets:

    • Google Play Store Distribution: The publishing process is similar to that of phone and tablet apps. You need to:
    • Create an APK and sign it with a release certificate.
    • Upload it to the Google Play Developer Console.
    • In the store listing information, navigate to the Android TV section and provide specific assets as required by the Play Store. The Play Store automatically recognizes your app as an Android TV app due to the CATEGORY_LEANBACK_LAUNCHER declaration in your manifest.
    • Amazon Fire TV Distribution: Since Fire OS 5, you can also distribute Android apps built with the Leanback Support library and Lollipop features through the Amazon App Store for Fire TVs. While specific compatibility details with Amazon Fire OS are beyond the scope of the sources, you can find documentation on the Amazon developer website. This allows you to reach a broader audience with potentially minimal modifications.

    In summary, publishing Android TV apps involves careful consideration of platform-specific requirements for the manifest, UI design (leveraging the Leanback Library is key here), search and discovery features, and controller support (for games). Adhering to the Android TV App Checklist and utilizing the developer consoles for both Google Play Store and Amazon App Store are the main steps for distributing your application to users.

    By Amjad Izhar
    Contact: amjad.izhar@gmail.com
    https://amjadizhar.blog

  • YouTube Content Creation: A Systematic Literature Review New Methods and Resources

    YouTube Content Creation: A Systematic Literature Review New Methods and Resources

    This academic paper presents a systematic review of research trends in audiovisual production for digital content creation on YouTube within Ibero-America from 2010 to 2021. The study analyzed scientific literature from databases like EBSCO and Scielo, focusing on resources, techniques used by creators, content preferences, audience interests, and content management strategies. Findings indicate a prevalent interest in new methods and resources in video production, with YouTube serving as a significant platform for diverse content and interaction. The research also examines factors influencing audience engagement and the professionalization of YouTube content creation.

    Study Guide: Audiovisual Production for Creating Digital Content on YouTube. Systematic Literature Review

    Quiz

    1. What was the primary objective of this systematic review? Briefly describe the timeframe and geographical focus of the research.
    2. Which four informative trends were identified in the research regarding audiovisual content creation on YouTube?
    3. According to the introduction, what are some of the advantages of the YouTube platform for creators of digital content?
    4. Describe the three main aspects of content creation on YouTube that León (2018) considered significant and relevant.
    5. What did the study by Muñoz (2018) indicate regarding the type of audiovisual production that tends to receive greater audience reactions on YouTube?
    6. How has the “YouTube Partners” program influenced content creators, according to the text?
    7. What were the main inclusion and exclusion criteria used for selecting articles for this systematic review?
    8. Name the three databases used to conduct the literature search for this review. What were the initial and final numbers of articles considered?
    9. According to the results, what were the most frequent years and the primary country of publication for the reviewed articles?
    10. Briefly outline the four phases involved in the process of creating a YouTube video according to León (2018), as described in the “Description of Resources and Techniques” section.

    Quiz Answer Key

    1. The primary objective of this systematic review was to document the research trends in audiovisual production applied to the creation of digital content for the YouTube platform. The research considered the period between 2010 and 2021 and focused on Ibero-America.
    2. The four informative trends identified were: resources and techniques employed by content creators, types of content and subject matter preferred by users, strategies for content management, and audiences interested in YouTube content creation.
    3. The advantages of YouTube include its large user base, its capacity to transfer different media formats, the space it offers for social interaction, and the accessible and intuitive tools for creating audiovisual content, allowing creators to manage their own channel and develop production skills.
    4. León (2018) considered the treatment of image and sound, the use of resources to communicate a message, and the correct understanding of the YouTube platform as the most significant and relevant aspects of content creation.
    5. The study by Muñoz (2018) indicated that there is a preference for live streaming as a type of audiovisual production because it generates greater reactions among the audience.
    6. The “YouTube Partners” program has benefited YouTube content creators through the “monetization” of their videos, having a positive impact on essential aspects such as how they communicate with their audience, the scenography, and the topics they address.
    7. The main inclusion criteria were articles published between 2010 and 2021, articles written in Spanish, and academic publications in scientific journals. The exclusion criteria included articles in a language other than Spanish and articles published outside the selected range of years.
    8. The three databases used were EBSCO, Scielo, and Dialnet. The initial search yielded 2,164 academic publications, which was refined to 15 articles after the screening process.
    9. The results indicated that the largest number of articles were published between 2017 and 2020, and the country where most texts were published was Spain, representing 80% of the reviewed articles.
    10. According to León (2018), the four phases of creating a YouTube video are: the creation of the idea and the “differentiat” strategy (pre-production), filming with a focus on technical aspects and scripts (production), editing and post-production using complementary software, and finally, displaying and dissemination, considering video titles, thumbnails, and popularity strategies.

    Essay Format Questions

    1. Discuss the evolution of research trends in audiovisual production for YouTube content creation between 2010 and 2021, as identified in this systematic literature review. What were the key areas of focus and how did they change over this period?
    2. Analyze the interplay between content creators, the YouTube platform’s features, and audience preferences in shaping the landscape of digital content creation, drawing upon the findings of this review.
    3. Critically evaluate the methodologies employed in the studies included in this systematic review. What are the strengths and limitations of focusing solely on Spanish-language academic publications within the Ibero-American context for understanding global trends in YouTube content creation?
    4. Based on the research trends identified, what recommendations can be made for aspiring or established YouTube content creators to enhance their production quality, audience engagement, and content strategy?
    5. Explore the economic and social implications of the professionalization of YouTube content creation, as suggested by the increasing focus on production quality, monetization strategies, and audience interaction highlighted in this literature review.

    Glossary of Key Terms

    • Audiovisual Production: The process of creating content that incorporates both visual and auditory elements, such as videos, films, and multimedia presentations.
    • Digital Content: Information or entertainment created and distributed through electronic media, including text, images, audio, and video.
    • YouTube: A global online video-sharing platform where users can upload, view, and interact with videos.
    • Systematic Review: A type of literature review that aims to identify, select, critically appraise, and synthesize all high-quality research evidence relevant to a specific research question.
    • Literature Review: A comprehensive summary of previous research on a topic, used to identify gaps, consistencies, and controversies in the existing body of knowledge.
    • Ibero-America: A region comprising the Spanish-speaking countries of the Americas and the Iberian Peninsula (Spain and Portugal). In this context, likely focuses on Spanish-speaking countries.
    • PRISMA Methodology: Preferred Reporting Items for Systematic Reviews and Meta-Analyses; a set of evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.
    • Content Creator: An individual or entity that produces and shares digital content online.
    • Monetization: The process of earning revenue from online content, often through advertising, sponsorships, or subscriptions.
    • Content Management: The process of plan

    YouTube Audiovisual Production Trends (2010-2021)

    Briefing Document: Research Trends in Audiovisual Production for YouTube Content Creation (2010-2021)

    Date: October 26, 2023 Source: “Audiovisual Production for Creating Digital Content on YouTube. Systematic Literature Review” (Excerpts from “01.pdf”, Proceedings of the 2022 International Conference on International Studies in Social Sciences and Humanities) Authors: Fabrizio Rhenatto Arteaga-Huaracaya, Adriana Margarita Turriate-Guzmán, and Melissa Andrea Gonzales-Medina

    1. Executive Summary:

    This document summarizes the key findings of a systematic literature review that aimed to document the research trends in audiovisual production applied to the creation of digital content for the YouTube platform in Ibero-America between 2010 and 2021. The review analyzed 10 scientific articles identified through a rigorous search process using the PRISMA methodology and various academic databases. The main themes identified include the resources and techniques employed by content creators, the types of content and subject matter preferred by users, strategies for content management, and audience interests in YouTube content creation. The study highlights the professionalization of YouTube content creation and the increasing sophistication in audiovisual production techniques.

    2. Main Themes and Important Ideas/Facts:

    • Growth and Importance of YouTube: The review acknowledges YouTube’s significant growth, noting that by the end of 2021, it had 2.56 billion users and facilitated the transfer of various formats, including live streaming, podcasts, and audiobooks. The platform offers tools for monetization and network management, making it an “ideal scenario for including transmedia and cross-media storytelling.”
    • Evolution of Audiovisual Production on YouTube: Research indicates increasing interest in new methods across all phases of video production. Technological progress has enabled producers to explore new tools for managing audiovisual material, especially in post-production. The audiovisual language of YouTubers is characterized by elements like shot type, appearance of people, image superimposition, dynamic editing, and the use of text, music, and effects.
    • Resources and Techniques Used by Content Creators: The initial trend analysis revealed four informative trends, with one focusing on the “resources and techniques employed by content creators.” According to León (2018), the creation of a YouTube video involves four phases: pre-production (idea creation), filming (technical aspects and scripts), post-production (editing and complementary software like Movie Maker or iMovie), and display/dissemination (titling, thumbnails, and SEO strategies using keywords and hashtags).
    • Types of Content and Subject Matter Preferences: Another identified trend pertains to the “types of content and subject matter preferred by users.” Avila and Avila (2019) suggest that media content posted on YouTube is a preferred option for the audience. Fabara et al. (2017) found that audiences generally prefer entertainment content such as comedy, video clips, and tutorials.
    • Strategies for Content Management: The review also explored “strategies for content management on YouTube.” Delgado (2016) conducted a study analyzing the audiovisual production of the “Extremoduro” music genre on YouTube, noting aspects such as technical quality and publication continuity as crucial for successful audiovisual material. Lindsay Stirling’s channel strategically uses the tools offered by YouTube to optimize the development and dissemination of video clips.
    • Audience Interest in YouTube Content Creation: Understanding “audience interests in YouTube content creation” is another key theme. Barredo, Pérez, and Aguaded (2021) found a relationship between education and the production of audiovisual content. The study by Muñoz (2018) on the viral video #LADY100PESOS indicated that production techniques focusing on viral potential aim to motivate public reactions and increase the content’s popularity.
    • Professionalization of YouTube Content Creation: The research suggests a trend towards greater professionalism in YouTube productions. Fabara, Poveda, Moncayo, Soria, and Hinojosa (2017) emphasized the ideal types of production for a YouTube channel are tutorials and web series. Ávila and Ávila (2019) point out that sensationalist content receives more public reaction, but the audience of the YouTube platform does not exclusively like sensationalist content, suggesting a need for positive viewer interactions.
    • Monetization and Creator Incentives: YouTube provides economic incentives for creators to improve their content development process. The “YouTube Partners” program has positively impacted content creators by offering monetization for their videos, considering aspects like audience, scenography, and topics.
    • Impact of Public Figures and Content Characteristics: Delgado (2016) noted that when a public figure is involved in audiovisual production, their acceptance and the number of views are not always related to the diversity of topics offered by the creator. Factors like the popularity of “YouTubers” can be an influential factor, although higher quality is still important.
    • Search Engine Optimization: The review highlights the importance of YouTube’s search engine, which tracks keywords in comments, descriptions, and titles. Orduña-Malea, Font-Julián, and Ontalba-Ruipérez (2020) found that a significant percentage of videos collected using specific queries were unrelated to the pandemic, indicating the challenges in targeted searches.
    • Training and Skill Development: Producing high-quality audiovisual pieces on YouTube often depends on the training received by the creator. Professional content creators tend to attract greater audience acceptance and intervention, often eliciting more opinions through comments.

    3. Key Quotes:

    • “By the end of 2021, YouTube had 2.56 billion users [1]. Its peak caused different media to transfer their formats to this platform in order to benefit from it [2]. This social network offers a space for storing and broadcasting videos, live streaming events, and adapting audiovisual content, such as audiobooks or podcasts through organization and editing tools. These include playlists, chat rooms, the possibility of monetizing views, or including network management strategies to strengthen the reach of audiovisual products, such as keywords or hashtags [3].”
    • “Research on trends and updates in the process of making digital content provides evidence that there is more interest in applying new methods in the various phases involved in the production of a video.”
    • “According to León (2018), the process of creating a YouTube video involves four phases. First, pre-production involves the creation of the idea and the ‘differentiat’ strategy [4]. Secondly, filming concentrates the technical aspects of recording and the application of scripts or guidelines. Thirdly, post-production is characterized by editing, using complementary software such as Movie Maker or iMovie. It also takes into account the implementation of visual and sound effects, such as the use of 2D graphics and various sound effects.”
    • “According to Avila and Avila (2019), media content posted on YouTube is one of the most preferred by the audience because it generates different reactions and obtains a large number of views; however, it receives both positive and negative interactions [9].”
    • “The ‘YouTube Partners’ program has benefited YouTube content creators through the ‘monetization’ of their videos because it has had a positive impact on essential aspects such as the way they communicate with their audience, the scenography, and the topics they address in their videos, intending to achieve the professional growth of their channels [11].”
    • “Regarding, Delgado (2016) tells us that the acceptance of audiovisual content and the number of views they get from the public is not always related to the diversity of topics that a creator may offer their channel, but, in most cases, an influential factor can be the popularity of these “YouTubers” [14].”
    • “Therefore, there may be cases in which videos that are not related to the search may appear. This scenario is evidenced in the research by Orduña-Malea, Font-Julián, and Ontalba-Ruipérez (2020), whose metric analysis revealed that “the YouTube API returned videos that were unrelated to Covid-19. 54.2% of the videos collected (using specific queries) were unrelated to the pandemic.” [15]”

    4. Conclusion:

    This systematic review provides valuable insights into the evolving landscape of audiovisual production for YouTube content creation in Ibero-America between 2010 and 2021. The findings highlight a clear trend towards greater sophistication and professionalism in content creation, influenced by user preferences, platform features, and the pursuit of audience engagement and monetization. The research underscores the importance of understanding various aspects of the production process, from technical execution to content strategy and audience interaction, for success on the YouTube platform. While the review identified key trends, it also notes the limited number of articles specifically focused on audiovisual production for YouTube within the defined scope and time frame, suggesting potential areas for future research.

    YouTube Content Creation: Ibero-American Audiovisual Production Trends

    Frequently Asked Questions: Audiovisual Production for Creating Digital Content on YouTube

    • What was the primary objective of the systematic literature review conducted? The main objective was to document the research trends in audiovisual production specifically applied to the creation of digital content for the YouTube platform within the Ibero-American context between 2010 and 2021.
    • What key methodological approach was used in this review to identify relevant research? The review adopted the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology for the selection of scientific literature. It involved searching the EBSCO, Scielo, and Dialnet databases using specific keywords related to audiovisual production and YouTube content creation, followed by a two-phase screening process based on predefined inclusion and exclusion criteria.
    • According to the research, what are the four informative trends observed in the resources and techniques used by YouTube content creators? The four informative trends identified were: resources and techniques employed by content creators, types of content and subject matter preferred by users, strategies for content management, and audiences interested in YouTube content creation.
    • What are some of the technical and artistic aspects that YouTube content creators need to consider in their audiovisual productions? Content creators need to consider technical aspects such as recording quality, editing, and the use of software. Artistically, they should focus on the type of shot, the number of people appearing in the video, the superimposition of images or videos, dynamic editing effects, the use of text, music, sound effects, silence, voiceover, and channel statistics to structure their content effectively.
    • How has YouTube as a platform influenced the diversification of digital content? YouTube has enabled the diversification of content by providing a platform where creators can produce and consumers can access a wide variety of videos according to their preferences. This necessitates that creators investigate different content formats and adapt their production style to resonate with their target audience.
    • What are some strategies YouTube creators employ for content management to increase audience engagement and visibility? Strategies include focusing on video titles, thumbnails, and popularity trends; optimizing content positioning through keywords and hashtags; maintaining a consistent publication continuity; and analyzing audience preferences to tailor content that motivates public reactions and generates positive viewer interactions.
    • What role do audience preferences and interactions play in the success of YouTube content creation? Understanding and catering to audience preferences is crucial for motivating viewership and achieving positive interactions, such as comments and likes. Creators need to be mindful of the type of content audiences prefer, the length of videos they are willing to watch, and the communication style that resonates with them to foster a connection and encourage engagement.
    • What were some of the limitations identified in the research regarding the available literature on audiovisual production for YouTube in the Ibero-American context? The research identified a limited number of articles and topics directly addressing audiovisual production for YouTube in the Ibero-American region between 2010 and 2021. Additionally, some articles lacked complete information, were not specific to the research question, or were published in languages other than Spanish.

    YouTube Audiovisual Production: Trends and Practices

    The source discusses audiovisual production primarily in the context of creating digital content for YouTube. This systematic literature review aimed to document research trends in audiovisual production applied to YouTube content creation in Ibero-America between 2010 and 2021.

    Here are some key aspects of audiovisual production discussed in the source:

    • Objective: The objective of research in this area is to understand and document trends in how audiovisual production is applied to create digital content specifically for the YouTube platform.
    • Platform Significance: YouTube has become a significant platform for transferring formats and adapting audiovisual content, offering tools for editing, monetization, and audience interaction. It is considered an ideal scenario for transmedia and cross-media storytelling, favoring interaction between users.
    • Production Process: The creation of a YouTube video involves several phases. León (2018) identified four phases:
    • Pre-production: Creation of the idea and the ‘differentiat’ strategy.
    • Filming: Focus on the technical aspects of recording and applying scripts or guidelines.
    • Post-production: Characterized by editing, using complementary software like Movie Maker or iMovie, and incorporating elements like 2D graphics and various sound effects. Conceptual videos might experience confusion during post-production if animation techniques and visual effects differences are not clearly marked.
    • Display and Dissemination: Involves choosing video titles, thumbnails, and popularity strategies, prioritizing content positioning through keywords and hashtags.
    • Technical Aspects: Research emphasizes the importance of technical elements in audiovisual production for YouTube. Pattier (2021) notes that the audiovisual language of YouTubers includes the type of shot, the type of angle, the number of people appearing in the video, the superimposition of images or videos, and the dynamic editing effects, as well as the use of text, music, effects, silence, voiceover, and channel statistics. Sedaño (2020) also points out the possibility of experimenting with a variety of audiovisual techniques, especially applied to the post-production phase.
    • Diversification of Production Types: YouTube has enabled the diversification of content, with a wide variety of production types available to help creators make audiovisual products, such as streaming, vlogs, and documentaries. Muñoz (2018) indicated a preference for live streaming as a type of audiovisual production because it generates more reactions.
    • Professionalism: There’s a trend towards greater professionalism in audiovisual production for YouTube. Professional creators tend to attract greater audience acceptance and intervention in their productions.
    • Economic Incentives: YouTube supports its creators through economic incentives to improve content development, benefiting content creators through the “YouTube Partners” program and the “monetization” of their videos. This has a positive impact on essential aspects like the way they communicate with their audience, the scenography, and the topics they address.
    • Audience Preferences: Understanding audience preferences is crucial. Fabara et al. (2017) found that audiences generally prefer content related to entertainment genres like comedy, video clips, and tutorials. They also tend to select YouTube videos to consume based on peer recommendations and abandon content that is long and less entertaining. Sensationalist content can receive more reactions but may not be preferred by the broader YouTube audience. The audience’s interest in creating and sharing audiovisual productions is also growing, particularly among young people.
    • Content Management Strategies: Effective content management includes understanding audience preferences for content and format, creating audiovisual pieces that the audience likes, and using language and expressions that create a connection with the user.
    • Training and Skills: Creating high-quality audiovisual pieces often depends on the training received by the creators. YouTubers who become prosumers are focusing on deepening the technical aspects necessary for planning, elaborating, and treating their productions.

    In conclusion, the source highlights that audiovisual production for YouTube is a dynamic field influenced by technological progress, evolving audience preferences, platform features, and a growing trend towards professionalization. Research in this area focuses on understanding these trends and documenting the strategies and techniques employed by content creators to produce engaging digital content.

    YouTube Digital Content Creation: Ibero-American Trends

    The source primarily discusses digital content within the specific context of audiovisual production for the YouTube platform. This systematic literature review aimed to understand the research trends related to the creation of digital content on YouTube in Ibero-America between 2010 and 2021.

    Here are some key aspects of digital content discussed in the source, specifically concerning YouTube:

    • Definition and Examples: In the context of YouTube, digital content refers to the audiovisual products created and uploaded to the platform. Examples include fan communities, animations, interviews, podcasts, tutorials, live streaming events, vlogs, and documentaries. The platform has enabled the diversification of such content.
    • Creation Process: The creation of YouTube digital content involves a structured process that can be broken down into phases. León (2018) identified four phases:
    • Pre-production: This involves the initial idea generation and the development of a unique strategy to make the content stand out.
    • Filming: This phase concentrates on the technical aspects of recording the audiovisual material, often utilizing scripts or guidelines.
    • Post-production: Editing is a crucial part of this stage, often utilizing software like Movie Maker or iMovie, and incorporating visual and sound effects. Meticulous editing is considered a key element for audience acceptance.
    • Display and Dissemination: This final phase includes selecting effective video titles and thumbnails, as well as employing popularity strategies that leverage keywords and hashtags to improve content visibility.
    • Influencing Factors on Content Creation: Several factors influence the creation and reception of digital content on YouTube:
    • Content Creator Techniques: Creators use various techniques to make their content viral and attract audience attention.
    • Audience Preferences: Understanding what the audience prefers is critical. Research suggests that audiences favor entertainment content like comedy, video clips, and tutorials. They also rely on recommendations and tend to abandon longer, less engaging content.
    • Platform Features: YouTube provides various tools and features for content creators, including editing capabilities, monetization options, and network management strategies. The platform’s design encourages interaction between users.
    • Professionalization: There is a growing trend towards greater professionalism in the creation of digital content on YouTube, with professional creators often achieving greater audience acceptance. This involves deepening technical skills in planning and production.
    • Economic Incentives: YouTube’s “Partner Program” and video monetization offer economic incentives for creators to improve their digital content, impacting aspects like communication, scenography, and topic selection.
    • Research Trends: Research in Ibero-America between 2010 and 2021 focused on documenting the trends in audiovisual production applied to the creation of digital content for YouTube. This included analyzing resources and techniques used by content creators, types of content and subject matter preferred by users, content management strategies, and audience interests.
    • Impact of Content: The quality and characteristics of digital content significantly impact its performance on YouTube. Factors like technical quality, production continuity, and strategic management are crucial for audience engagement and the success of a YouTube channel.

    In summary, the source emphasizes that digital content on YouTube is a multifaceted concept shaped by the platform’s features, content creator practices, evolving audience preferences, and a growing trend towards professional production. Research in this area seeks to understand these dynamics and identify the key factors influencing the creation and consumption of audiovisual digital content on YouTube.

    The YouTube Platform: Content Creation and Consumption

    The source extensively discusses the YouTube platform as a central element in the creation and consumption of digital content. Here’s a breakdown of the key aspects of the YouTube platform highlighted in the text:

    • Significance and Scale: By the end of 2021, YouTube had 2.56 billion users, marking it as a significant platform for transferring various content formats. It serves as a crucial social network that offers space for storing and broadcasting videos, live streaming events, and adapting audiovisual content like audiobooks and podcasts. As of June 23, 2022, records indicated 572,011 videos and 114,282 channels on YouTube.
    • Tools and Features for Creators: YouTube provides advantages to creators by offering them their own channel with accessible and intuitive tools for creating audiovisual content. These tools include options for editing, the possibility of monetizing views, and network management strategies to strengthen the reach of audiovisual products through keywords and hashtags. The platform has enabled the diversification of content, making it possible for users to create and consume videos according to their preferences.
    • Ideal Scenario for Content Creation: The source positions YouTube as an ideal scenario for including transmedia and cross-media storytelling due to its ability to favor interaction among users and the creation of derivative content that extends a story.
    • Influence on Audiovisual Production: YouTube has significantly influenced audiovisual production by becoming a key platform for content development. Content creators on digital platforms like YouTube have implemented structured procedures in their audiovisual content production. The platform allows for experimentation with audiovisual techniques, particularly in the post-production phase.
    • Audience Engagement and Preferences: YouTube’s structure allows content creators to shape their content in ways that attract and retain their audience. Features like frequency of uploads, YouTuber personality, integration with other networks, YouTube channel tools (home panel, channel art, comments, playlists, community, store function), all help to structure content for the audience. The platform generates different views and obtains both positive and negative interactions from the audience. Youth nowadays are prominent consumers of new types of content on YouTube, with music videos and promotional videos being among the most prominent.
    • Economic Ecosystem: YouTube supports its creators through economic incentives for improving content development. The “YouTube Partners” program has benefited content creators through the “monetization” of their videos, impacting how they communicate, their scenography, and the topics they cover. This suggests that the platform fosters a professionalization of content creation.
    • Content Management on YouTube: YouTube is considered a database where the public can access a large amount of content. YouTube channels are seen as ideal formats to attract a target audience. Effective content management on the platform involves keeping up with audience preferences for content and format, creating appealing audiovisual pieces, and using communication styles that resonate with users.
    • Research Focus: Academic research, as highlighted in the systematic review, focuses on understanding the trends in audiovisual production applied to the creation of digital content on the YouTube platform. This includes analyzing the resources and techniques used, content preferences, management strategies, and audience interests within the YouTube ecosystem.

    In summary, the YouTube platform is presented as a powerful and multifaceted environment for the creation, distribution, and consumption of audiovisual digital content. It offers creators a range of tools and opportunities while being significantly shaped by audience preferences and evolving production techniques. The platform’s economic model and features encourage a degree of professionalism among content creators. Academic research recognizes YouTube as a crucial area for studying trends in digital content and audiovisual production.

    YouTube Content Creators: Production, Strategies, and Professionalization

    The sources provide several insights into Content Creators, particularly those who produce audiovisual material for the YouTube platform.

    • Definition and Role: The source indicates that content creators on digital platforms like YouTube are actively involved in the production of their audiovisual content. YouTube has become a preferred social network for content creators to disseminate their material.
    • Production Process: Research on trends in making digital content reveals that content creators follow a production procedure for their audiovisual content. León (2018) outlines four phases that content creators typically engage in: pre-production (idea generation and strategy), filming (technical recording), post-production (editing and effects), and display/dissemination (titling, thumbnails, and popularity strategies using keywords and hashtags). Meticulous and attractive editing is considered a key element for audience acceptance of a content creator’s work.
    • Techniques and Strategies: Content creators employ various techniques to make their content viral and attract audience attention. They focus on creating content that will motivate public reactions. For example, some YouTubers use production techniques that emphasize sensationalist content to generate more reactions. Content creators need to investigate the different types of content available on YouTube to focus their production style and adapt to what is most appreciated by their audience. They also need to consider the preferences of the audience regarding audiovisual formats, such as “click bait” content and short films. Furthermore, content creators are increasingly focusing on deepening the technical aspects necessary for planning, elaboration, and treatment of their productions.
    • Motivation and Goals: For many seeking to become YouTubers, the goal is to manage their own channel and have accessible tools for creating audiovisual content. Content creators are often interested in learning the steps involved in production. They also strive to create quality content and understand the importance of elements like titles and thumbnails. The proper selection of discourse, type of production, and format can represent a content creator’s ability to interact with the subject matter effectively, leading to greater audience acceptance.
    • Audience Relationship: Content creators must consider the audience throughout the creation process. They need to understand the types of content and format the audience prefers to create appealing audiovisual pieces. Understanding audience preferences and interactions (positive and negative) is crucial for a content creator’s success. Some research indicates that audiences prefer content related to entertainment, such as comedy, video clips, and tutorials. Content creators aim to achieve positive viewer interactions.
    • Professionalization: The source suggests a trend towards greater professionalism among content creators. This involves acquiring a correct understanding of the YouTube platform and focusing on technical quality and production continuity. Professional creators tend to attract greater audience acceptance and intervention in their productions.
    • Economic Factors: YouTube supports its creators with economic incentives through the “YouTube Partners” program, which allows for video monetization. This program has positively influenced how content creators communicate, their scenography, and the topics they address, encouraging professional growth.

    In our previous discussion, we noted that YouTube provides various tools for content creators, including editing capabilities and monetization options [Me]. The source further elaborates on how these features and the economic ecosystem influence content creators’ practices and motivations. The increasing professionalism observed in content creation on YouTube aligns with the platform’s evolution into a significant space for digital content production, as we discussed earlier [Me].

    YouTube Digital Content Creation: Ibero-American Research Trends

    The sources highlight several research trends concerning audiovisual production for digital content creation on the YouTube platform. The primary objective of the systematic review presented in the source is to document these research trends in Ibero-America between 2010 and 2021.

    Here are the key research trends identified:

    • Resources and Techniques Employed by Content Creators: Research has focused on the tools, software (like Movie Maker and iMovie), and methods that content creators utilize in the creation process. This includes examining the four phases of video production as outlined by León (2018): pre-production, filming, post-production, and dissemination. Studies also analyze the application of scripts or guidelines and the use of animation techniques and visual effects. Furthermore, research delves into the increasing focus on the technical aspects of planning, elaboration, and treatment of YouTube productions.
    • Types of Content and Subject Matter Preferred by Users: Another significant research trend involves understanding the kind of content that YouTube users prefer and consume. This includes analyzing the popularity of different formats like “click bait” content, short films, music videos, and promotional videos. Studies also investigate if users prefer entertainment genres such as comedy, video clips, and tutorials. Research in this area aims to identify what type of content is most likely to attract and retain audience attention.
    • Strategies for Content Management on YouTube: Research also explores the various approaches and tactics used to manage content effectively on the YouTube platform. This includes understanding how creators use keywords and hashtags, titles and thumbnails, and strategies for the development and dissemination of video clips to optimize channel growth. The role of channel features like the home panel, channel art, comments, and playlists in structuring content for the audience is also considered.
    • Audiences Interested in YouTube Content Creation: Understanding the characteristics and preferences of the audience is a crucial research trend. Studies analyze the relationship between the production of videos and audience education and how factors like YouTuber personality and engagement influence audience interaction. Research also examines how sensationalist content can generate more reactions from the public and the impact of audience feedback (positive and negative) on content creators.

    Beyond these four main trends, the source highlights other areas of research:

    • Influence of YouTube on Audiovisual Production: Research explores how YouTube has impacted traditional media and the evolution of audiovisual language. Studies analyze how the platform enables experimentation with audiovisual techniques, especially in post-production.
    • Economic Aspects of Content Creation: The impact of YouTube’s monetization programs, like the “YouTube Partners” program, on content creators’ practices and professional growth is also a subject of research.
    • YouTuber Language and Communication: Some research focuses on the specific communication styles and language used by YouTubers, including the analysis of aggressive expressions in gameplay narration and the effectiveness of different discourse types.
    • YouTube as a Social and Marketing Platform: Research also investigates how companies and brands use YouTube and other social networks for their communication and marketing efforts.

    The systematic review itself utilized a rigorous methodology, adapting the PRISMA guidelines for the selection and analysis of relevant scientific literature published between 2010 and 2021. This methodological approach is a research trend in itself, indicating an academic interest in systematically understanding the evolving landscape of audiovisual production on YouTube. The findings of this review contribute to understanding the key areas that have been the focus of scholarly inquiry in this field.

    By Amjad Izhar
    Contact: amjad.izhar@gmail.com
    https://amjadizhar.blog

  • From Thrift Store to CEO

    From Thrift Store to CEO

    Two young women, Laura and Kendra, have contrasting approaches to business. Laura, initially struggling with her thrift store upcycling venture, perseveres after receiving encouragement and pivots to Instagram, achieving great success. Kendra, relying on family wealth, initially mocks Laura but ultimately fails in her own business endeavors. The narrative highlights the importance of passion and adaptability in entrepreneurship versus relying solely on financial backing. Laura’s journey showcases perseverance and finding the right platform, while Kendra’s story serves as a cautionary tale.

    Live Clam: Passion, Perseverance, and Profit

    Business Ventures and the Power of Passion: A Study Guide

    Quiz

    Answer each of the following questions in 2-3 sentences.

    1. Why does Kendra mock Laura’s initial business idea?
    2. What is Laura’s initial business plan for her clothing business?
    3. What advice does Laura’s teacher, Miss Jacobs, give her about pursuing her business idea?
    4. What role does Instagram play in Laura’s eventual business success?
    5. How does Kendra’s financial background influence her view of success and business?
    6. What specific challenge does Laura face when she initially tries to sell her products?
    7. What does Laura mean when she states, “When you do what you love, the money will always follow?”
    8. How does Laura demonstrate her resourcefulness and creativity in the context of her business?
    9. What unexpected twist occurs when Kendra comes to apply for a job at Laura’s company?
    10. How does the final scene of the excerpt tie back to the initial conflict of the story?

    Answer Key

    1. Kendra mocks Laura’s idea because she views thrift store clothing as “old” and “smelly” and believes it’s disgusting to wear used clothes. She also doubts Laura’s ability to make money selling inexpensive items.
    2. Laura initially plans to buy used clothing, customize it by adding “bling” and other creative touches, and then sell the unique pieces for a profit, initially on social media and at events.
    3. Miss Jacobs advises Laura to not give up, to continue doing what she loves, and that if she is passionate about her work, the money will follow. She encourages Laura to focus on her love for her product, not just sales figures.
    4. Instagram allows Laura to showcase her products to a much wider audience. This is what transforms her business from slow sales to a booming success, leading to a huge social media following.
    5. Kendra’s wealthy background leads her to judge others based on their financial resources, causing her to prioritize high-end products and doubt the viability of low-cost businesses. She believes that success comes from large investments and having access to wealthy contacts.
    6. Laura initially struggles to generate significant sales because her Facebook posts don’t reach a wide enough audience, demonstrating the challenge of marketing effectively. People are not readily buying her products, resulting in her questioning her entire business strategy.
    7. Laura’s belief reflects the idea that passion and dedication to one’s work will eventually lead to financial success. This philosophy suggests that doing something one loves is more important than the initial financial gains.
    8. Laura’s resourcefulness and creativity are demonstrated through her ability to take used clothes and transform them into desirable and trendy items by adding bling and making them “cute.” She is able to create unique pieces and profit with minimal overhead costs.
    9. Kendra is shocked to find out that Laura owns the company where she applies for a job and discovers that Laura’s business became very successful, while her own “funded” business is unsuccessful, leading to her searching for jobs.
    10. The final scene highlights the shift in their fortunes and the difference in business strategy and overall success. Laura’s success demonstrates that passion, creative vision and perseverance can triumph, while Kendra’s initial arrogance is proven wrong.

    Essay Questions

    Consider these questions in developing a more thorough understanding of the material. Do not supply answers to these questions.

    1. Analyze the contrasting attitudes of Laura and Kendra towards business and success. How do their backgrounds and personalities shape their approaches and ultimately their results?
    2. Discuss the role of social media and marketing in the success or failure of Laura’s business. What does the narrative suggest about the evolving landscape of business and commerce?
    3. Explore the theme of perseverance in the story. How does Laura’s experience demonstrate the importance of resilience and adaptability in the face of challenges?
    4. Examine the role of mentorship and encouragement in Laura’s journey. How do the characters of Miss Jacobs and the first customers demonstrate the impact of belief and support?
    5. Reflect on the message conveyed by the narrative in relation to wealth, status and success. Does it prioritize hard work and passion over financial resources? Support your view.

    Glossary of Key Terms

    • Thrift Store: A store that sells used goods, typically clothing, furniture, and household items, often at low prices.
    • Bling: Flashy jewelry or decorative accessories, often sparkly or shiny, used to embellish clothing or other items.
    • Customized: Modified or altered to suit individual preferences or needs, often involving unique additions or designs.
    • Business Plan: A detailed proposal outlining a company’s objectives, strategies, target market, financial projections, and operational plans.
    • Pitch (to investors): A presentation made to potential investors, aiming to convince them to invest money into a business or project.
    • Birkin: A luxury handbag made by Hermès, known for its high price and exclusivity.
    • Social Media: Digital platforms that allow users to create, share content, and interact with each other, such as Facebook and Instagram.
    • Profit: The financial gain or surplus resulting from a business transaction after subtracting costs and expenses.
    • Cosmetics Company: A business that produces and sells products related to personal care, makeup, and beauty.
    • Uber X: A ride-sharing service offering standard vehicles at a more affordable price than luxury options.
    • Senior Manager: A high-level leadership role within a company, often overseeing a team or department.
    • Live Clam: Name of the company that Laura eventually owns, showing her ultimate business success.
    • Resourcefulness: The ability to find quick and clever ways to overcome difficulties and solve problems.
    • Perseverance: Continued effort and dedication to achieve a goal, despite obstacles or setbacks.
    • Resilience: The capacity to recover quickly from difficulties, showing toughness and adaptability.

    Passion, Perseverance, and the Power of Instagram

    Okay, here’s a detailed briefing document analyzing the provided text:

    Briefing Document: Entrepreneurship, Social Media, and the Power of Passion

    Overview:

    This text presents a narrative contrasting two approaches to entrepreneurship, highlighting the importance of passion, perseverance, and adapting to evolving market trends. It follows the journeys of Laura, who starts a business customizing and selling used clothing, and Kendra, who relies on her father’s wealth to pursue her business dreams. The story underscores how initial setbacks and skepticism can be overcome with a genuine love for one’s work and the willingness to embrace new platforms and strategies.

    Main Themes and Key Ideas:

    1. Passion vs. Pragmatism: The story sets up a clear contrast between Laura, who is driven by her passion for creativity and upcycling, and Kendra, who is focused on financial success and status.
    • Laura: Her motivation is evident when she says, “I do this because I love it, not because I’m here to make money.” This approach contrasts sharply with Kendra’s business plan, driven by a desire to make “$2 million in sales my first year.”
    • Kendra: Kendra’s approach is pragmatic and focused on financial success, leading to her initially dismissing Laura’s business as “used trash.” Her ultimate goal is status and luxury items like a Birkin bag.
    • Quote: “I was taught that when you do what you love the money will always follow” – This phrase, repeated by Laura and later by Miss Jacobs, embodies the core theme of the text.
    1. The Power of Perseverance: Laura faces numerous challenges, including initial skepticism, lack of sales, and public ridicule from Kendra. Her ability to rebound from these setbacks showcases the power of perseverance.
    • Initial struggles: Laura experiences slow sales on Facebook, feeling like her business is a “total failure”. This leads to her considering returning all her thrift store finds.
    • Turning point: Miss Jacob’s encouragement, echoing Laura’s own philosophy (“when you do what you love the money will always follow”), re-ignites her motivation. This moment is a crucial one in which she goes from giving up to being excited and determined.
    1. Adaptability and Embracing New Platforms: The story underscores the importance of being adaptable and willing to embrace new strategies. Laura’s turning point is her willingness to pivot to Instagram.
    • Facebook Fail: Her initial attempt to sell her products on Facebook failed miserably, with many stating that “no one buys anything on Facebook”.
    • Instagram Success: Laura’s discovery of Instagram is a pivotal moment, allowing her to reach a wider audience and ultimately achieve phenomenal success. “People all over the world start to discover her work” showing how a social media platform can make or break a company.
    1. Social Media’s Impact on Business: The narrative shows how social media can be a powerful tool for both failure and success.
    • Initial Ineffectiveness: Laura’s initial reliance on Facebook proves fruitless, showcasing the importance of selecting the right platform.
    • Transformative Power of Instagram: Instagram provides the visual platform needed to showcase her products and cultivate a global customer base. This highlights the impact of the right platform, and how powerful a platform like Instagram can be.
    1. The Dangers of Elitism and Condescension: Kendra’s character serves as a cautionary tale about the dangers of elitism and dismissing others’ ideas.
    • Early dismissal: Kendra consistently belittles Laura’s business model, expressing disgust at the idea of selling used clothes. She refers to Laura’s merchandise as “trash”.
    • Karma: Kendra, in the end, is reduced to applying for a job at Laura’s hugely successful company, learning a lesson in humility.
    • Quote: “you know 10 years from now while you’ll still be buying you stuff I’ll be shopping at a maze and buying Birkins” exemplifies Kendra’s elitist mindset and disdain for others not in her position.
    1. Defining “Business”: The text touches on what constitutes a real “business.”
    • Kendra’s View: Kendra initially dismisses Laura’s endeavor, arguing “you don’t have a business, you just have a dream,” because Laura hadn’t yet made a sale. This highlights her practical, metrics driven view of business.
    • Laura’s reality: Laura, despite initially not generating much revenue, was clearly building a business through her actions, which later translates into actual sales and success.
    1. Importance of Genuine Connection: The story suggests the importance of genuine connection with one’s customers and products.
    • Laura’s Passion: She was passionate about what she did and as the business grew it wasn’t just about making money. It was about doing what she loved and sharing her creativity with others.
    • Authenticity: Laura’s authenticity resonated with consumers and led her to success.

    Character Dynamics:

    • Laura: The protagonist, characterized by her creativity, passion, and resilience. She is willing to take a risk and learns from both successes and failures.
    • Kendra: The antagonist, embodies privilege, pragmatism, and a condescending attitude. She ultimately learns a lesson about the value of hard work and passion.
    • Miss Jacobs: She serves as a mentor figure, providing timely encouragement to Laura and reinforces the core message of the story.

    Narrative Arc:

    The narrative follows a classic arc of challenges and triumphs, from Laura’s initial spark of an idea to her eventual success. It also demonstrates the cyclical nature of success and failure, as Kendra’s initial position of superiority is reversed by the end of the story.

    Conclusion:

    This text offers a compelling look at the entrepreneurial journey. It celebrates the value of passion, resilience, and adaptability in achieving success. It cautions against arrogance and elitism, and highlights the power of social media as a business tool. The text’s most enduring message is that when you are passionate and love what you do, money and success will always follow. This narrative encourages budding entrepreneurs to persevere, adapt to market changes, and pursue business ventures that truly resonate with them.

    Upcycled Business FAQs

    Frequently Asked Questions about Starting a Business from Upcycled Goods

    1. What are some of the key benefits of starting a business focused on upcycled or repurposed goods?
    • Upcycling can be very cost-effective, as the base materials are often purchased at low prices or thrifted. This allows for a greater potential for profit margins when the items are customized and resold. It can also be a good way to create unique products as well as offer an alternative to fast fashion, appealing to consumers looking for sustainability and individuality. This model can attract those interested in creative and unconventional designs.
    1. What challenges might one face when launching a business selling customized upcycled items?
    • Initially, one may experience slow sales and struggle to find an audience. It can also be difficult to compete with well-established brands and businesses. Also, there could be skepticism about the value of used goods. Finding a suitable platform to reach an audience is also a challenge, as well as managing a high volume of items through manual processes of acquisition, customization and sale.
    1. How important is passion and a love for the craft when starting a business like this, especially when facing challenges?
    • Passion is crucial. It acts as a motivator to persist through difficulties such as slow sales, and can help cultivate creativity. The belief in one’s work, even in the face of skepticism, is needed. A love for the process can make the endeavor more than just about profit, providing intrinsic satisfaction even before monetary success is achieved.
    1. How can social media play a vital role in a business selling upcycled goods?
    • Social media platforms, particularly image-focused ones like Instagram, are essential for showcasing unique upcycled designs to a wider audience. They allow for global reach, enabling entrepreneurs to connect with customers beyond their local area. A strong social media presence can lead to rapid growth and brand recognition by sharing visually compelling content, attracting attention and potential customers.
    1. What are some of the pitfalls of relying solely on personal funding (like family money or investments) versus bootstrapping a business?
    • Relying solely on family funding may not develop needed business acumen through experience. It also creates an atmosphere where if money isn’t handled wisely there may be a loss of investment with little acquired in the process. Bootstrapping, while slower, can lead to a stronger foundation, more flexibility and a better understanding of the market. It also allows the entreprenuer to take ownership of their growth. Additionally, funding from a third party can potentially come with additional benefits like business mentorship and partnerships.
    1. How does a focus on creating quality unique products stand against the pursuit of rapid profit generation?
    • Focusing on unique products built out of love may initially yield smaller profits than strategies focused solely on financial gain. However, businesses based on a genuine passion and unique items can develop a dedicated customer base. When customers connect to the values of the product they are more likely to develop brand loyalty which can create long-term sustainability. This kind of business can attract those seeking something special.
    1. What role can mentorship or advice play in an entrepreneur’s journey, especially when considering changing direction?
    • Mentorship can offer crucial support, encouragement and valuable perspective, especially when a business faces setbacks. Wise advice can help entrepreneurs avoid giving up too soon and provide insights on new strategies, like moving to more suitable platforms (as seen with Instagram). Guidance can steer entrepreneurs to focus on core business values and help them pivot without sacrificing their core passion.
    1. How can initial setbacks and criticism be a catalyst for growth and learning in a business journey?
    • Negative feedback and early failures can be a necessary learning experience. It may reveal that certain strategies don’t work, thus promoting the business owner to try new approaches, find gaps in the market and ultimately refine the overall business model. When handled correctly, setbacks can strengthen resilience and improve business sense. Overcoming these hurdles can often lead to greater success as long as persistence and focus are maintained.

    From Thrift Store to CEO

    Okay, here’s the timeline and cast of characters based on the provided text:

    Timeline of Events

    • Initial Encounter at the Thrift Store/School: Kendra and Laura run into each other. Kendra expresses her distaste for thrifting and used clothes. Laura reveals her plan to start a business customizing and selling thrifted clothing and accessories.
    • Business Plan Pitch: Laura pitches her business plan in class, detailing her idea of customizing used clothing. Kendra is openly dismissive of her plan. Laura expresses her belief that “when you do what you love the money will follow.”
    • Laura’s First Sale: After school, Laura sets up a small table and is ridiculed by Kendra, who later trashes all her products. Despite this, Laura makes her first sales when a couple of friends purchase necklaces.
    • Early Business Struggles: Laura begins selling her products on Facebook, but sales are very slow, leading to discouragement and the belief that her business will fail.
    • Laura’s Decision to Quit: Laura decides to give up on her business and goes to the thrift store to return her unsold items.
    • Miss Jacobs’ Intervention: Miss Jacobs, one of Laura’s former teachers, encourages her to keep going and reminds her of her “do what you love” motto. She also mentions the app “Instagram” as a new platform Laura could try to reach more customers.
    • Business Success on Instagram: Laura tries selling on Instagram. Her business rapidly grows as she gains followers and customers and finds success.
    • Laura’s Cosmetics Company: Laura’s business expands into a cosmetics company, and she partners with her fiance.
    • Kendra’s Job Interview: Kendra, now broke, interviews for a senior manager position at Live Clam, Laura’s company. She reveals that her dad is mad at her for losing his money.
    • Kendra’s Rejection and Offer of Assistant Position: Kendra is rejected for the senior manager position, which is instead offered to someone else. Laura is now the owner of Live Clam, not just an employee. Laura owns the company with her fiance. Kendra ends up being offered an assistant position in the end.
    • Kendra Meets Laura Again: Laura reminds Kendra that when you do what you love, the money will follow, even though they both use Uber X as a mode of transportation.
    • The Interview: Laura is shown starting the interview with Kendra for the assistant position.

    Cast of Characters

    • Laura: The protagonist. Initially a thrifter and student, she has a dream of starting a business customizing used clothes. She faces discouragement but perseveres and becomes a successful business owner with her own clothing and cosmetics company. She holds to the mantra, “when you do what you love, the money will always follow.”
    • Kendra: The antagonist. A wealthy student who is dismissive of thrifting and Laura’s business aspirations. She is initially confident in her own financial advantages but experiences a reversal of fortune, ending up broke and seeking a job.
    • Miss Jacobs: Laura’s teacher who encourages her not to give up. She reminds Laura of her philosophy of doing what you love and also suggests Instagram as a potential selling platform. She is an advocate for Laura’s success.
    • Laura’s Fiancé (Mentioned only): Laura’s business partner, also co-owner of her company, Live Clam.
    • Unnamed classmates: Several classmates buy a necklace at Laura’s first sale. There is also a general reference to students talking about Laura’s business when Miss Jacobs recommends the app Instagram.

    Let me know if you have any other questions.

    Thrift Store Empire: From Rags to Riches

    A business that centers around thrift store items is the main focus of the sources [1-5]. Here’s an overview of key aspects of the thrift store business concept presented:

    • Sourcing: The business model involves acquiring used items, like clothing and shoes, from thrift stores [1-3]. These items are typically low cost [1].
    • Customization: The thrifted items are then customized to enhance their appeal. This includes adding “bling” or other creative alterations [1-3]. For example, a shirt collar is removed and made into a necklace [2].
    • Sales Strategy:The initial idea was to sell the customized items in person at a table [2, 3].
    • The business plan initially included a social media component with Facebook [3, 4], which proved unsuccessful [3, 4].
    • The business owner later started using Instagram [4], which was significantly more successful [4].
    • Pricing and Profit: The goal is to sell the customized items at a profit [2]. In one instance, an item was purchased used and resold for $7 [2].
    • Challenges:Initially, the business owner faced skepticism from others who thought the concept was not viable [1-3].
    • The business struggled with slow sales when relying on Facebook [3, 4].
    • The business owner considered giving up due to a lack of early success [3, 4].
    • Success Factors:The business owner was passionate about the products [4].
    • The business owner was told by a wise person that “when you do what you love the money will always follow” [2, 4, 5].
    • The business owner found a better selling platform in Instagram [4].
    • The business owner was told to not give up [4].
    • Outcomes: The thrift store business ultimately became very successful, with a large following and many customers [4]. The business owner also started a cosmetics company [4, 5].

    Overall, the sources show the development of a thrift store based business from a struggling idea to a successful venture, highlighting the importance of passion, perseverance, and the right sales platform.

    Upcycled Fashion Business Model

    Customized clothing is a key element of the business described in the sources [1-4]. Here’s an overview of how it’s presented:

    • Source Material: The clothing that is customized is sourced from thrift stores [1-4]. This allows the business to acquire items at a low cost.
    • Customization Techniques: The business owner uses various methods to alter and enhance the thrifted clothing [1, 2, 4]. These techniques include:
    • Adding “bling” [1, 2].
    • Cutting off collars of shirts to make necklaces [2].
    • Purpose of Customization: The customization is done to make the clothing more appealing and unique [1-4]. The goal is to transform used, ordinary clothing into “cute” items [1, 3].
    • Examples of Customized Items: The sources provide several specific examples of customized clothing:
    • Shirts with added “bling” [2].
    • Shirts with the collars removed to make necklaces [2].
    • Customized shoes [1, 4].
    • Business Model: The customized clothing is the core product of the business [1, 2]. The business owner purchases used clothes, customizes them, and resells them at a higher price [1, 2].
    • Sales:The business owner initially attempted to sell the customized clothing in person and on Facebook [2, 3].
    • The business owner later found success selling the customized clothing on Instagram [4].
    • Pricing: The customized items are sold for a profit, one example given was that a customized necklace was sold for $7 [2].
    • Significance: The concept of customized clothing is central to the business’s identity and success. The transformation of used clothing into unique, desirable items is the foundation of the business [1, 2, 4]. The owner’s creativity and skill in customization are key to the business’s appeal [2, 4].

    Social Media Sales: Facebook vs. Instagram

    Social media sales play a significant role in the business described in the sources, though the effectiveness of different platforms varies dramatically. Here’s an overview of how social media sales are presented:

    • Initial Attempt with Facebook: The business owner initially plans to use Facebook as a sales platform [1]. She intends to post pictures of her customized clothing on Facebook to find customers [1].
    • This strategy proves to be unsuccessful. The sources indicate that “no one buys anything on Facebook” [1].
    • The business owner’s Facebook posts do not attract customers, leading to slow sales and feelings of failure [2, 3].
    • The lack of sales on Facebook is a significant factor in the business owner considering giving up [2].
    • Success with Instagram: The business owner is encouraged to try selling on Instagram [3].
    • She starts posting photos of her customized items on Instagram, which leads to a significant increase in visibility and customer interest [3].
    • The business owner’s Instagram following explodes, and she gains a large customer base [3].
    • Instagram becomes a successful platform for selling her products [3].
    • Contrast between Platforms:
    • Facebook is presented as an ineffective platform for the business, with the statement “no one buys anything on Facebook” [1]. The business owner’s experience supports this, as sales are slow and lead to her considering giving up [2, 3].
    • Instagram is presented as a highly effective platform, leading to a large increase in sales, a huge following, and the overall success of the business [3].
    • Significance:The source material underscores the importance of choosing the right social media platform for sales [1, 3].
    • The business owner’s experience demonstrates that a social media strategy is not a guarantee of success; the platform must align with the target audience and product [1, 3].
    • The transformation from struggling with Facebook to succeeding with Instagram highlights the impact that a suitable platform can have on a business [1, 3].

    In summary, while social media is essential for the business described in the sources, the success of social media sales is heavily dependent on the chosen platform. The initial attempts to use Facebook to sell the customized clothing proved ineffective, while the shift to Instagram resulted in remarkable growth and widespread success for the business [1, 3].

    From Thrift Store to Birkin Bag

    Business success, as depicted in the sources, is a multifaceted journey marked by initial struggles, adaptation, and eventual triumph [1, 2]. Here’s a detailed look at the factors contributing to the business’s success:

    • Initial Challenges and Setbacks:
    • The business owner starts with a concept of customizing and reselling thrift store items [3].
    • The business encounters significant skepticism from others, who doubt its viability [3].
    • The initial sales strategy of selling in person and on Facebook proves ineffective [1, 4]. The sources specifically state that “no one buys anything on Facebook” [1, 4].
    • The lack of sales leads the business owner to consider giving up [1, 2].
    • She experiences slow sales and feels like her business is a failure [1].
    • Pivotal Moments and Turning Points:
    • A key turning point occurs when the business owner receives encouragement not to give up [1, 2].
    • She is reminded of the principle “when you do what you love the money will always follow” [2, 4, 5]. This principle is mentioned multiple times and seems to be important to the business owner’s success [2, 4, 5].
    • The suggestion to try selling on Instagram is critical [2].
    • This transition to Instagram as a sales platform marks a significant shift in the business’s trajectory [2].
    • Factors Contributing to Success:
    • Passion: The business owner is passionate about her products and what she does [2].
    • Perseverance: Despite early setbacks, she continues to pursue her idea [1, 2].
    • Adaptability: She is willing to shift her sales platform from Facebook to Instagram [1, 2].
    • Platform Choice: Instagram proves to be a successful platform for selling her products [2].
    • Unique Product: The customized items are “cute” and unique, attracting a customer base [1, 3, 4].
    • Creative Customization: The business owner’s ability to transform thrift store items into desirable pieces is a key factor in her success [3, 4].
    • Outcomes of Success:
    • The business experiences an “explosion” in followers and customers [2].
    • The business becomes highly successful, surpassing initial expectations [2].
    • The business owner expands her business to include a cosmetics line [2].
    • The business owner is eventually able to afford luxury items like a Birkin bag [4, 5].
    • The business owner’s journey highlights the importance of aligning passion with a viable sales strategy and the transformative power of perseverance [2, 5].
    • Contrast with Another Character:
    • The success of the business contrasts sharply with another character’s journey, who, despite having financial backing from her father, ends up applying for an entry level job with the company [3, 5]. This underscores the idea that financial backing alone doesn’t guarantee success [5]. The sources emphasize that doing what you love is a key component to success [2, 4, 5].

    In summary, the business’s success is not a linear path but a result of overcoming challenges, adapting to new opportunities, and a continued commitment to the business owner’s passion. The effective use of Instagram as a sales platform plays a vital role in the business’s growth and overall success [2].

    From Struggle to Success: A Business Journey

    Financial struggles are a significant aspect of the business journey depicted in the sources, particularly in the initial stages. Here’s an overview of the financial challenges and how they were addressed:

    • Initial Low-Cost Model: The business starts with a low-cost model, sourcing used clothing and shoes from thrift stores [1]. This allows for minimal initial investment.
    • Pricing and Profit:The business owner aims to sell the customized items at a profit. For example, she sells a customized necklace for $7 [2].
    • However, the business owner mentions that she is “not too worried” about the money, because she believes that if you do what you love, the money will follow [2].
    • Slow Sales and Low Revenue:The initial sales efforts on Facebook are unsuccessful, leading to slow sales [3]. The source material states that “no one buys anything on Facebook” [3].
    • As a result, the business owner makes very little money in the first few months, making “less than like a few hundred” dollars [4].
    • This lack of revenue leads to feelings of failure and the consideration of giving up [3, 4].
    • The slow sales make the business owner feel like her business is a “total failure” [3].
    • Returns:The business owner attempts to return her unsold merchandise to the thrift store, a sign of the severity of her financial struggles [3].
    • She is initially told that the store does not usually accept returns [4].
    • Contrast with Another Character:Another character in the sources has financial backing from her father [1]. However, she doesn’t find success with her own business venture and ends up applying for an entry-level job at the successful business, highlighting that money alone does not guarantee success [5].
    • Overcoming Financial Struggles:The business owner is encouraged not to give up and to continue doing what she loves [4].
    • The business owner shifts her sales platform to Instagram, which leads to a significant increase in sales and revenue [4].
    • The business eventually becomes very successful, demonstrating how financial challenges can be overcome [4].
    • Long-term Financial Success:The business owner’s initial financial struggles contrast with her eventual financial success, as she is eventually able to afford luxury items like a Birkin bag and start her own cosmetics company [2, 4, 5].
    • She also is able to hire a senior manager and an assistant, demonstrating the long term financial success of her company [5].
    • She references that “when you do what you love the money will always follow,” as the cause of her financial success, referencing the earlier advice she received [2, 4, 5].

    In summary, the sources portray the business owner’s initial financial struggles as a significant obstacle, with slow sales and low revenue. The shift to a more effective sales platform, coupled with her passion and perseverance, led to a transformation from a financially struggling business to a highly successful and lucrative enterprise.

    Contrasting Business Approaches: Laura vs. Kendra

    Laura and Kendra have contrasting business approaches, both in their initial ideas and their subsequent execution, which is highlighted in the sources. Here’s a detailed comparison:

    Laura’s Business Approach:

    • Concept and Ideology: Laura’s business is centered around customizing used clothing and reselling it [1]. Her core belief is that “when you do what you love the money will always follow” [2-4]. This suggests that her approach is driven by passion and creativity rather than solely by profit [2].
    • Product and Customization: Laura sources her materials from thrift stores, which allows her to keep her costs low [1]. She customizes these items, making them “cute” by adding “bling,” cutting off collars to make necklaces and customizing shoes [1-3]. Her customization focuses on transforming used items into unique, desirable pieces [2].
    • Initial Sales Strategy: Laura initially tries selling her customized items in person and on Facebook [2, 5]. This approach is unsuccessful, with the sources stating, “no one buys anything on Facebook” [3, 5].
    • Adaptability: Despite early setbacks, Laura adapts by shifting her sales strategy to Instagram, where she finds success [3].
    • Motivation: Laura is primarily motivated by her passion for creating and customizing clothing and her belief that if you do what you love the money will follow [2, 3].
    • Financial Strategy: Laura focuses on low-cost inputs from thrift stores and selling her creations at a modest profit [1, 2]. She is not initially focused on making large sums of money.
    • Long Term Success: Laura’s business eventually becomes very successful, as she gains a large following on social media, starts her own cosmetics company, and can afford luxury goods [3, 4].

    Kendra’s Business Approach:

    • Concept and Ideology: Kendra’s approach is characterized by a more traditional business mindset. Her initial plan is to secure funding through investors or her wealthy father and to create a large business [1]. The sources do not indicate that she has any specific passion for fashion or creating; she seems more motivated by money and status.
    • Product and Strategy: Kendra does not create anything. Instead, her plan appears to be based around having a large budget and buying expensive items that are already considered desirable.
    • Initial Sales Strategy: Kendra does not begin selling any products herself, but rather pitches her idea to investors [1].
    • Financial Backing: Kendra’s business plan relies on funding from her father [1].
    • Motivation: Kendra appears to be motivated by financial success, luxury, and status, such as owning Birkin bags [2].
    • Contrast with Laura: Unlike Laura, Kendra is not driven by a love of her product or creativity; rather, she seems primarily concerned with profit and status [1, 2].
    • Long Term Success: Kendra’s business plan fails and she ends up applying for an entry level position at Laura’s company [4].

    Key Differences:

    • Motivation: Laura is driven by her passion for creation and her belief that money will follow if you do what you love [2-4], while Kendra is driven by money and status [1, 2].
    • Product: Laura focuses on unique, customized, low-cost items, whereas Kendra’s plan is focused on purchasing expensive, high-end luxury items [1-3].
    • Financial Approach: Laura starts with a low-cost model, while Kendra relies on financial backing [1].
    • Adaptability: Laura demonstrates adaptability by changing her sales platform [3], while Kendra’s initial plan fails, and she ends up needing to find a job with Laura’s company [4].
    • Success: Laura’s passion and creativity ultimately lead to a highly successful business [3, 4], while Kendra’s initial business plan fails [4].

    In summary, Laura’s approach is characterized by creativity, adaptability, and a belief in the power of passion, whereas Kendra’s is more traditional, focused on financial backing and status, and ultimately less successful. The sources emphasize that Laura’s success is due to her passion and that money will follow if you do what you love [2-4].

    Laura’s Early Marketing Struggles

    Laura initially marketed her products through a combination of in-person sales and online efforts, which are detailed in the sources:

    • In-Person Sales: Laura began by selling her customized clothing and accessories directly to people. The sources mention that she set up her items on “some beastro tables” [1]. This suggests a local, small-scale approach to sales, likely in a physical location like a market or pop-up shop.
    • Facebook: In addition to in-person sales, Laura also attempted to market her products on Facebook [1]. This was an early attempt to reach a broader audience online. However, this approach proved to be unsuccessful. The sources specifically state that “no one buys anything on Facebook” [1, 2].
    • Initial Challenges: Laura experienced significant challenges in these initial marketing efforts. Her sales were slow and she made very little money, leading her to feel like her business was a “total failure” [2]. These struggles led her to consider giving up [2].
    • Shifting Strategies: Eventually, Laura shifted her marketing strategy away from Facebook and in-person sales, due to their ineffectiveness [3]. This marked a turning point, where she moved to Instagram and found success [3].

    In summary, Laura’s initial marketing approach involved a combination of local, in-person sales, and online efforts through Facebook. These initial efforts, however, were largely unsuccessful [1, 2]. It was only after transitioning to Instagram that her business began to experience growth and success [3].

    Laura’s Entrepreneurial Triumph: From Rags to Riches

    Laura’s ultimate business success was the result of several interconnected factors, evolving from initial struggles to eventual triumph, as described in the sources and our previous conversation. Here’s a breakdown of the key elements that contributed to her success:

    • Passion and Perseverance: Laura’s business was fueled by her passion for customizing used clothing and her belief that “when you do what you love the money will always follow” [1-3]. Despite facing significant challenges, she persevered, demonstrating a commitment to her vision [4].
    • Adaptability and Pivotal Changes:
    • Laura’s initial marketing strategies, which included in-person sales and Facebook, were ineffective [1, 2, 4]. The sources specifically mention that “no one buys anything on Facebook” [2, 4].
    • A key turning point was the suggestion to use Instagram as a sales platform [2]. This pivot was crucial for her business growth.
    • Laura’s ability to adapt her sales strategies based on performance was essential to her success.
    • Unique Product and Creative Customization: Laura’s ability to transform used items into desirable, unique products was a key factor [1, 5]. Her customizations were described as “cute” and included adding “bling”, cutting off collars to make necklaces and customizing shoes [1, 2, 4, 5]. The unique nature of her products attracted a customer base.
    • Effective Use of Social Media: The shift to Instagram proved to be transformative [2]. By posting photos of her customized products on Instagram, she was able to reach a much wider audience. This led to an “explosion” in her following and customer base [2]. Her business expanded far beyond her local contacts and she gained worldwide recognition.
    • Contrast with Kendra’s Approach: Unlike Kendra, whose business plan relied on financial backing and a traditional approach, Laura’s focus on creative customization, low-cost inputs, and using social media to market her unique products led to success. [1, 5]. Kendra’s plan ultimately failed [3]. The sources emphasize that financial backing alone does not guarantee success [3].
    • Long-Term Vision: Her eventual success was far beyond what she initially imagined [2]. She eventually expands her business to include a cosmetics line [2, 3].

    In summary, Laura’s success was not a result of one single factor, but rather a combination of her passion, perseverance, adaptability, creative customization, and effective use of Instagram. Her journey highlights the importance of aligning one’s passion with a viable sales strategy and the transformative power of perseverance. The key to her success was her shift to Instagram and the combination of that platform with her unique products, which resulted in a massive increase in followers and customers.

    Laura’s Evolving Sales Strategy

    Laura initially planned to sell her customized items through a combination of in-person sales and online efforts [1].

    • In-person sales: Laura started by setting up her items on “some beastro tables,” which suggests a local, small-scale approach, likely in a physical location such as a market or pop-up shop [1]. This allowed her to directly interact with potential customers.
    • Facebook: Laura also attempted to market her products on Facebook, trying to reach a broader audience online [1]. However, this approach proved unsuccessful, with the sources stating, “no one buys anything on Facebook” [1, 2].

    These initial marketing efforts proved to be challenging, with slow sales and low revenue, making Laura feel like her business was a “total failure” [1, 2]. Eventually, she shifted her strategy away from these methods after experiencing little to no success [1]. It was not until she began selling on Instagram that she found success [2].

    Kendra’s Disdain for Laura’s Thrifting Venture

    Kendra initially reacted to Laura’s business idea with disdain and skepticism, expressing strong doubt about its potential for success [1, 2]. Here’s a breakdown of her reactions:

    • Dismissiveness and mockery: Kendra was dismissive of the idea that buying used items could be “cute” [1]. She mocked the concept of thrifting, stating, “you think that buying other people’s used stuff is cute it’s really not that bad,” and calling the clothes “old smelly clothes” that are “disgusting” [1].
    • Doubt about Profitability: Kendra expressed doubt about the profitability of Laura’s business, questioning Laura’s ability to make a substantial profit selling items for a low price [2]. When Laura states she is selling necklaces for $7, Kendra says, “are you kidding me?” [2]. Kendra’s business plan has her making “2 million in sales my first year” while she believes Laura’s plan will never generate any significant revenue [2].
    • Belief in Failure: Kendra repeatedly stated her belief that Laura’s business would never work, telling her, “that little business idea of yours is never going to work” and “no one’s going to want to buy your used trash” [1, 2]. She implied that Laura should give up on her business idea, suggesting Laura should “return all that stuff and get your…$12 back” [1].
    • Emphasis on Status and Material Wealth: Kendra’s reactions stemmed from her focus on status and material wealth. Her statement that in “10 years from now while you’ll still be buying you stuff I’ll be shopping at a maze and buying Birkins” indicates her focus on expensive luxury goods, and demonstrates a lack of understanding or appreciation for Laura’s approach [2].
    • Condescending Behavior: Kendra’s comments were often condescending and mean spirited, as demonstrated when she threw all of Laura’s items in the trash [2]. Despite saying she was not trying to be mean, she still acted in a way to undermine Laura [2].

    In summary, Kendra’s initial reaction to Laura’s business idea was marked by disbelief, mockery, and a general lack of support. She believed that Laura’s thrifting approach was inherently inferior and doomed to fail due to her own focus on status and luxury goods. Kendra’s dismissive attitude highlighted a contrast between her traditional, money-driven approach and Laura’s more creative, passion-driven one.

    Kendra’s Ironic Career Trajectory

    Kendra’s ultimate career outcome was that she ended up working for Laura’s company, despite her initial plans to be a successful business owner, as detailed in the sources. Here’s a breakdown of her career trajectory:

    • Initial Business Plan Failure: Kendra’s initial business plan, which relied on funding from her wealthy father and a traditional approach to business, did not succeed [1, 2]. Her plan was focused on obtaining a large budget to purchase expensive items such as Birkin bags, rather than creating a product or service herself [1, 2].
    • Job Seeking: After her business plan failed, Kendra needed to find employment. She applied for a senior manager position at Laura’s company, Live Clam, not realizing that Laura was the owner [3]. This shows a significant shift from her initial plan to own her own company to needing a job.
    • Interview with Laura: Kendra was interviewed by Laura, the owner of the company, who revealed that she was the founder and CEO [3]. This surprised Kendra, who previously mocked Laura for her business idea [2, 3].
    • Rejection for Senior Manager Position: Despite interviewing for the senior manager role, Kendra was informed that the position was no longer available because someone else had already accepted the offer [3].
    • Accepting an Assistant Position: Kendra was then offered and accepted an assistant position at Laura’s company, as it was the only position available [3]. This was a significant step down from the senior manager position she had initially applied for, and was a far cry from her original plan to be a successful business owner [1, 3].
    • Working for Laura: Kendra now works directly for Laura, whose business she previously doubted and ridiculed [2, 3]. This outcome is ironic, given her earlier comments and behavior [2].

    In summary, Kendra’s ultimate career outcome is that she was hired as an assistant at Laura’s company, after her own business plan failed and she was rejected from a more senior position. Her change in career path is a direct contrast to her initial plans and her attitude towards Laura’s business. This shows a significant shift from her initial goals and a form of poetic justice for her previous behavior toward Laura.

    The Original Text

    if you weren’t going to buy those shoes I would have they’re so hot yeah and I love how they go with your new bag yeah I was actually thinking that I could wear it with this one outfit excuse me oh I’m I’m I’m so sorry Laura hi Kendra wait do not tell me that you shop at the thrift store uh yeah I just picked up some really cute things C what cute it’s a way of saying cute oh got it let me get this straight you think that buying other people’s used stuff is cute it’s really not that bad you save a ton of money plus most of the stuff has hardly been worn look no I could never wear someone else’s old smelly clothes that’s disgusting seriously ew yeah I can’t believe you’re going to wear all that oh it’s not for me to wear it’s for me to sell what do you mean I’m starting a business where I customize clothes and shoes and I make them look cute like check these out I bought got them used and then I added my own bling to them and You’ never even know they cost less than $10 right wait you’re wearing $10 shoes you’ve got to be joking Kendra just spent over $600 on a pair oh our Uber black just got here I’d love to stay in chat but we got to go and I’m just looking out for you but that little business idea of yours is never going to work you might just want to return all that stuff and get your I don’t know $12 back so that’s my plan to own my own clothing store and who knows maybe I’ll even own my own makeup line one day it’s very impressive and I’d imagine expensive how do you plan to fund it all oh well that’s why I created this business plan to pitch to investors and if they say no I’ll just ask my dad he’s got a lot of money that makes sense all right well great job [Applause] Kendra let’s see how about Laura awesome job yeah you killed it thanks this should be good whenever you’re ready okay so I’ve actually already started my own business where is there something funny about that Kendra yeah when you say you’ve already started your own business have you sold anything yet not yet but I plan to after school today when I set up my uh I’m not talking about the future I’m talking about now have you made a single Dollar in sales um no then you don’t have a business you just have a dream don’t get them confused hey be nice sorry Laura you can continue okay so I have a dream to start a business where I take used clothes make them cute and then sell them for example take this shirt I cut off the collar add some bling to it and with that I made this necklace wow that’s so creative yeah I can’t believe you made that thanks and the best part is that I can sell it for just $7 and still make a profit $7 are you kidding me what’s wrong with that my business plan has me making 2 million in sales my first year what are you going to make selling $7 necklaces oh well I’m not too worried about that I was taught that when you do what you love the money will always follow I actually really like that with that mentality She’ll always be shopping at the thrift store Kendra what I can’t tell the truth sorry you didn’t get a chance to finish Laura but excellent presentation all right class we’ll pick back up tomorrow you know 10 years from now while you’ll still be buying you stuff I’ll be shopping at a maze and buying Birkins I’m really happy for you [Music] Kendra hey don’t let Kendra get to she thinks because her dad has money she can treat people however she wants personally I love your idea really thanks hey guys you want to come check out this stuff sorry I don’t have time okay no worries hey any of this stuff interests you I’m all right [Laughter] thanks well let me guess no one’s bought anything yet not yet but I think someone will soon so this is your business idea huh to sell a bunch of used stuff on some beastro tables well just to start I plan on posting everything on Facebook and then getting my customers on there no one buys anything on Facebook yeah that’s never going to work well with the way social media seems to be going I think they will one day it doesn’t even matter no matter where you sell no one’s going to want to buy your used trash what’s your problem Kendra I haven’t done anything to you why do you keep being so mean a you think I’m being mean I’m sorry that wasn’t my intention at all if I was trying to be mean I’d probably do this hey why would you do that because that’s where it belongs just like that necklace you’re wearing it’s all [Music] trash hey is everything okay no Kendra just threw all this stuff in the trash are you serious I can’t stand her you should just ignore everything she says I try to but sometimes I can’t help but think what if she’s right maybe my idea will never work hey don’t say that I really like your business thanks but like Kendra said it’s more of a dream than a business I haven’t sold anything that’s not true what do you mean I’ll take one bling necklace please really okay thank you here thanks for being my first customer so I guess that makes me a second I’ll take that one thanks um okay two three four five six shoot I’ve only got $6 oh that’s okay don’t worry about it are you sure I feel bad I know you don’t make much off of these no it’s it’s fine I do this because I love it not because I’m here to make money thanks I can’t wait to wear it and just like that Laura made her first sales she would take things she bought from the thrift store and customize them to make them look cute or should I say cute [Music] she started taking pictures of all of her pieces and posted them on social media excited to find some customers she sold her products everywhere she could and things seemed to be going great that is until she realized that her sales were really slow people didn’t really seem to be [Music] interested and nobody was seeing her Facebook posts Laura started feeling like her business was a total failure and then one day she even decided to give up great what else could go wrong well well well look who it is what do you want Kendra I’m not in the mood why do you seem so upset I don’t want to talk about it now please leave me alone let me guess that little dream of yours didn’t work out no it didn’t that’s why I’m here returning all this stuff is that what you wanted to hear a I’m sorry can’t say I’m surprised though I tried to warn you well I guess I should have listened hey don’t beat yourself up over it I’ll tell you what my dad just agreed to fund my business so after it takes off maybe you can come work for me I’m sure I’ll need an assistant to carry all my Birkin bags hey uh can I help you yeah I’d like to return all this stuff if possible oh um we actually don’t typically accept returns but since you shop here a lot maybe I can go talk to the manager just give me one second thank you [Music] Laura hey hi Miss Jacobs what are you doing here I just came to look for some things after hearing your presentation a few weeks ago I’ve become quite hooked on thrifting check out this belt I bought from here can you believe this only cost $4 that’s amazing thanks so what are you doing here uh I I’m just returning some stuff you don’t need all of that for your business things didn’t work out like I’d hoped so I decided to give up no don’t say that you barely even started why would you give up so soon in the past couple of months I’ve made less than like a few hundred my whole plan to sell things on Facebook let’s just say was a bad idea no one buy anything well let me ask you something are you passionate about the product you make well yeah and do you love what you do I did I mean I guess I still do why and you shouldn’t give up because a wise person once told me when you do what you love the money will always follow wow I can’t believe I almost forgot that don’t worry about how much you sold just keep going and everything will fall into place you’re right thanks Miss Jacobs I’m I’m never going to forget this [Music] anytime so good news my manager agreed to let you return everything actually I’ve changed my mind but thank you okay well have a good day bye Miss Jacobs wait one more thing what’s up I’ve been hearing a lot of my students talk about this new app it’s called Instagram I’m not sure if you heard of it but supposedly it’s all the rage maybe you should try selling there I’ll definitely check it out thanks you’re welcome and by the way that belt is so cute thank you with the new level of excitement and motivation Laura decides to keep going she keeps making all kinds of new products including customized shoes shoes necklaces and [Music] clothes this time when she takes photos she posts them on Instagram and people all over the world start to discover her work over time her following explodes she ends up with millions of followers and thousands of customers all loving her photos and excited to buy her products her business became even more successful than she could have ever imagined and eventually she even started her own Cosmetics company with a little bit of help of course hi guys hey babe but what do you think of this palette a it’s cute you mean CA Ella let tell Daddy it’s so cute do you like it yeah here’s your strawberry is saw you refresher oh my gosh thank you so much your 12:30 is in the conference room waiting oh perfect do you mind putting that in my office and grabbing my bag I left it in there oh yeah no problem thank you so much okay bye El M have a kiss a love you bye good luck hey what are you doing here oh hey I’m here to apply for a job I didn’t realize you worked here but I thought you started your own business oh well that didn’t exactly go according to plan my dad’s still pretty mad at me for losing all of his money oh no I’m so sorry to hear that it’s fine I’m actually here to interview for a senior manager position here at live clam if I end up getting it who knows I might still end up being your boss oh well hi hi you must be the owner I’m Kendra it’s so nice to meet you I was actually just talking to your assistant here oh um she’s not my assistant she isn’t no she’s my boss Laura owns this company with her fiance you do yeah thank you so much wait is that a Birkin mhm do you mind if I of course wow I’ve never held one of these in person before it’s so nice I guess selling $7 products from the thrift store really worked well you can say that gosh sorry to interrupt hi I just got a message from HR Sally accepted the senior manager offer oh wow so that means that this position uh unfortunately this position is no longer available oh my gosh I’m so sorry I’ll totally reimburse you for your time I mean is there any other position available just the assistant position oh no Kendra would not no that’s fine honestly I’ll take whatever I can get oh okay then have a seat oh here’s your bag thank you so I’m dying to know how did you get to where you are well it’s like I’ve always said when you do what you love the money will always follow I remember hey babe uh no rush but our Uber X will be here soon okay thank you so much I’ll be right there okay thanks with all your guys success you still ride Uber X I mean there’s nothing wrong with being Thrifty well not for everything right all right so let’s get this interview started [Music]

    By Amjad Izhar
    Contact: amjad.izhar@gmail.com
    https://amjadizhar.blog

  • Al-Riyadh Newspaper, April 2, 2025: Oil Production Expectations, Gaza, Bologna Children’s Book Fair, Poetry

    Al-Riyadh Newspaper, April 2, 2025: Oil Production Expectations, Gaza, Bologna Children’s Book Fair, Poetry

    This collection of Arabic news articles from April 2025 covers a diverse range of topics. Several pieces discuss economic matters, including oil production expectations, a decrease in non-oil exports in Saudi Arabia, and European chemical industry concerns over energy costs and competitiveness. Other articles focus on regional conflicts and humanitarian issues, notably the situation in Gaza and an Israeli strike in Lebanon. Cultural and social events are also reported, such as Eid al-Fitr celebrations, a Saudi delegation’s participation in the Bologna Children’s Book Fair, and the enduring nature of poetry across generations. Finally, the sources include sports news, updates on Saudi football teams, and information on the Saudi Green Initiative.

    Understanding the Provided Sources: A Study Guide

    Quiz: Short Answer Questions

    1. According to the text, what is Saudi Arabia’s strategic investment in the cyber domain focused on regarding children?
    2. What was the primary expectation for the OPEC+ ministerial meeting mentioned in the article, and what factor introduced uncertainty?
    3. In Geneva, what did the Saudi representative deliver on behalf of 75 countries regarding children in the cyber environment?
    4. What is the Saudi Green Initiative, and what is its primary goal concerning the Kingdom’s future?
    5. What did the Governor of the Jazan region emphasize to the citizens and residents who came to greet him for Eid al-Fitr?
    6. Describe one way in which the exchange of Eid greetings has changed in modern times, according to the article.
    7. According to the report mentioned, what was a significant observation regarding Saudi Arabia’s non-oil exports in the fourth quarter of 2024?
    8. What is the objective of the project aimed at improving the governance of Special Purpose Entities (SPEs) in Saudi Arabia’s capital market?
    9. What is the “Mashaer” project by Umm Al Qura Development Company, and what is its intended impact in Mecca?
    10. What was the primary reason for the stabilization of oil prices mentioned in the article despite concerns about global supply?

    Quiz: Answer Key

    1. Saudi Arabia’s strategic investment in the cyber domain is focused on the protection of children in cyberspace. This includes uniting global efforts to respond to and enhance international cooperation against challenges facing children online.
    2. The primary expectation for the OPEC+ ministerial meeting was a continuation of oil production increases, similar to the agreed-upon increases for April and May. However, threats from U.S. President Donald Trump regarding tariffs on Russian crude and potential action against Iran introduced uncertainty due to concerns about their impact on global economic growth and energy demand.
    3. In Geneva, the Saudi representative delivered a statement on behalf of 75 countries at the Human Rights Council, emphasizing the importance of building capacities to protect children in the cyber environment. This was done in the context of Saudi Arabia hosting the first global summit of its kind on child cybersecurity.
    4. The Saudi Green Initiative is a bold step and a fundamental part of the Kingdom’s future plans. Its primary goal is to envision a more sustainable future where the environment is an essential element of the state’s forward-looking strategies, aiming to combat climate change and protect the planet.
    5. The Governor of the Jazan region emphasized the leadership’s care and attention for all citizens and residents, the importance of the citizen’s role in national development and progress, and the need to preserve the nation’s security, safety, capabilities, gains, and the values upon which the country was founded based on Islamic principles.
    6. One way the exchange of Eid greetings has changed is the shift from traditional family visits and handwritten cards to digital methods such as social media posts, short text messages, and voice calls. This has made the process faster and more convenient, especially for those living far from family.
    7. A significant observation regarding Saudi Arabia’s non-oil exports in the fourth quarter of 2024 was a decrease in total non-oil re-exports, reaching 35.2% of total exports. Additionally, non-oil exports via air outlets also decreased compared to the previous quarter.
    8. The objective of the project is to enhance the attractiveness of Special Purpose Entities (SPEs) as investment vehicles by improving their governance and facilitating their procedures for issuing debt instruments and sukuk. This aims to develop the debt and sukuk market, diversify issuances, and boost liquidity.
    9. The “Mashaer” project by Umm Al Qura Development Company is a large-scale development project in Mecca that includes residential units, hotels, retail spaces, and pedestrian walkways, along with infrastructure and services. It aims to be a high-quality addition for visitors of the Grand Mosque and residents of Mecca, representing a shift in real estate development.
    10. The primary reason for the stabilization of oil prices was the easing of concerns about the impact of the trade war on global growth, despite threats from President Trump regarding tariffs and Iran. The market’s focus shifted to awaiting the OPEC+ meeting for clearer direction.

    Essay Format Questions

    1. Analyze the various initiatives and strategies mentioned in the provided texts that demonstrate Saudi Arabia’s commitment to environmental sustainability and its role in global environmental leadership.
    2. Discuss the economic diversification efforts highlighted in the articles, focusing on the trends in non-oil exports and the development of new sectors like technology and tourism, as evidenced by the “Mashaer” project.
    3. Examine the role of technology and innovation, particularly in the cyber domain and artificial intelligence, as areas of both opportunity and challenge for Saudi Arabia and the global markets, according to the provided sources.
    4. Compare and contrast the traditional and modern ways of celebrating Eid al-Fitr as depicted in the texts, and discuss the significance of these celebrations in Saudi Arabian society.
    5. Evaluate the factors influencing the global oil market as presented in the articles, including the actions of OPEC+, geopolitical tensions, and the potential impact of economic policies on supply and demand.

    Glossary of Key Terms

    • الفضاء السيبراني (al-faḍāʾ al-saybarānī): Cyberspace; the interconnected digital environment, including the internet and computer networks.
    • أوبك+ (ʾūbik+): OPEC+; an alliance of oil-producing countries, including the 13 members of OPEC and 10 other major non-OPEC oil-exporting nations.
    • قمة (qimmah): Summit; a meeting of heads of state or government, usually to discuss important issues.
    • المبادرة الخضراء السعودية (al-mubādara al-khaḍrāʾ al-saʿūdiyyah): The Saudi Green Initiative; a national program aimed at enhancing environmental sustainability, reducing emissions, and increasing reliance on clean energy.
    • عيد الفطر (ʿīd al-fiṭr): Eid al-Fitr; the Islamic holiday that marks the end of Ramadan, the month of fasting.
    • الصادرات غير البترولية (al-ṣādirāt ghayr al-bitrūliyyah): Non-oil exports; goods and services exported by a country that are not crude oil or petroleum-based products.
    • المنشآت ذات الأغراض الخاصة (al-munshaʾāt dhāt al-ʾaghrāḍ al-khāṣṣah): Special Purpose Entities (SPEs); legal entities created for a specific, narrow, and well-defined purpose.
    • حوكمة (ḥawkamah): Governance; the system by which a company or organization is controlled and operated.
    • مخزونات الخام (makhzūnāt al-khām): Crude oil inventories; the total amount of unrefined petroleum products held in storage.
    • الذكاء الاصطناعي (al-dhakāʾ al-iṣṭināʿī): Artificial Intelligence (AI); the theory and development of computer systems able to perform tasks that normally require human intelligence.
    • التصحر (al-taṣaḥḥur): Desertification; the process by which fertile land becomes desert, typically as a result of drought, deforestation, or inappropriate agriculture.
    • محميات طبيعية (maḥmiyyāt ṭabīʿiyyah): Nature reserves; protected areas of land or sea, designated for the conservation of biodiversity and natural resources.
    • التنوع البيولوجي (al-tanawwuʿ al-biyūlūjī): Biodiversity; the variety of life in the world or in a particular habitat or ecosystem.
    • الإعمار (al-ʾiʿmār): Reconstruction/Development; the process of rebuilding or developing something that has been damaged or is underdeveloped.
    • اكتتاب (ʾiktitāb): Subscription (in finance); the process of offering new shares for sale to the public or existing shareholders.

    Briefing Document: Analysis of “Al Riyadh” Newspaper Excerpts (April 2, 2025)

    This briefing document summarizes the main themes, important ideas, and key facts presented in the provided excerpts from the April 2, 2025 issue of the Saudi newspaper “Al Riyadh.” The excerpts cover a wide range of topics, reflecting the diverse interests and ongoing developments within the Kingdom and the broader region.

    I. Domestic Developments and Initiatives:

    • Technological Advancement and Space Exploration: The Kingdom emphasizes its leading role in space technologies and exploration. Significant progress has been made in recent years, with projects aiming to improve the quality of life and boost economic development through the application of space technologies in sectors like health, agriculture, and education.
    • Key Facts: The Kingdom has launched several satellites, including “SaudiSat,” for communications and internet services. It is developing spacecraft like “Al Najm” for space exploration and has launched the “Saudi Space Program” to enhance scientific research and technology development.
    • Quote: “تعتبر المملكة من الدول الرائدة يف جمال الفضائية، والتقنيات الفضاء شهدت تطورًا كبيرًا يف السنوات األخيرة ويف العديد من املجاالت، حيث مت يف هذا السبيل إطالق املشاريع والربامج الفضائية التي تعمل على تحسني جودة احلياة وتعزيز التنمية االقتصادية، إضافة إىل تعزيز البحث العلمي يف جمال الفضاء.” (Page 16)
    • Key Fact: Saudi Arabia achieved a global first by launching a Saudi research experiment to study and analyze the “Space Microbiome” on the polar orbit of the International Space Station. This is described as a significant step in space exploration and research.
    • Protection of Children in Cyberspace: The Kingdom considers protecting children in cyberspace a strategic investment. This is highlighted by the launch of the first global summit of its kind dedicated to child protection in the cyber domain, initiated by Crown Prince Mohammed bin Salman.
    • Key Idea: The summit aims to unify international efforts to address the challenges facing children in the cyber world.
    • Key Fact: Saudi Arabia, represented by Ambassador Abdulmohsen bin Khothaila, delivered a statement on behalf of 75 countries at the Human Rights Council in Geneva, emphasizing the importance of building capabilities to protect children online.
    • Environmental Sustainability and the Saudi Green Initiative: The “Saudi Green Initiative” is presented as a bold step and a fundamental part of the Kingdom’s future plans, aiming for a more sustainable future and positioning Saudi Arabia as a leading force in combating climate change.
    • Key Ideas: The initiative embodies a comprehensive strategic vision and the Kingdom’s commitment to fighting climate change, protecting the planet, achieving comprehensive development, and building a better future for future generations.
    • Key Facts: The initiative aligns with Vision 2030 and includes planting 10 billion trees across the Kingdom and managing vast desert afforestation projects using modern technologies. It aims to restore vegetation cover, combat desertification, and improve air quality.
    • Quote: “وتعد مبادرة السعودية الخضراء خطوة جريئة يف تاريخ اململكة، حيث أصبحت البيئة جزءًا أساسيًا من اخلطط املستقبلية للدولة، وبفضل هذه املبادرة أصبح من املمكن تصور مستقبل أكثر استدامة لألجيال القادمة.” (Page 16)
    • Key Fact: The anniversary of the Saudi Green Initiative is commemorated on March 27th each year to highlight environmental achievements and support future plans. The Kingdom aims to reduce carbon emissions by 60% by 2030.
    • Focus on Citizen Welfare and Development: Prince Mohammed bin Nasser bin Abdulaziz, the Governor of Jazan, emphasized the leadership’s attention to the well-being of citizens and residents, highlighting the importance of the citizen’s role in national development, security, and the preservation of the Kingdom’s values based on Islamic principles.
    • Eid Al-Fitr Celebrations and Traditions: The excerpts detail various celebrations across the Kingdom for Eid Al-Fitr, highlighting the joy and social cohesion the occasion brings. Traditional customs like exchanging greetings, which have evolved with social media, and the enduring popularity of “Ma’amoul” (filled cookies) are mentioned. Villages are noted for retaining their special charm during Eid, emphasizing the simplicity and family gatherings of the past.
    • King Salman’s Relief Efforts: The King Salman Humanitarian Aid and Relief Centre continues its efforts to support affected and needy people in several countries.
    • Key Fact: In Syria, the center distributed 976 food baskets and 976 health bags in Jindires, benefiting 5,856 individuals affected by the earthquake. Aid distribution also occurred in northern Syria.
    • Key Fact: The center also distributed dates and provided iftar meals to fasting individuals in Argentina, benefiting 54,000 people, along with distributing copies of the Quran.
    • King Fahd Cultural Center in Argentina: The center is described as a prominent Islamic landmark in Latin America, having served the Muslim community in Argentina since 2000. It includes a mosque, school, library, and cultural center.
    • Development Projects in Makkah: The removal of unplanned neighborhoods in Makkah, adjacent to the central area, is underway. This is seen as enhancing services for visitors to the holy city and residents. However, observations within these areas revealed the informal commercial activity and the challenges of aligning it with Makkah’s global status.

    II. Economic Matters:

    • Oil Market Anticipation of OPEC+ Meeting: The oil market is awaiting the OPEC+ ministerial committee meeting scheduled for Saturday (April 5th, inferred). Sources indicate OPEC+ is likely to proceed with previously agreed production increases (135,000 barrels per day for May, similar to April).
    • Key Trend: Oil prices stabilized near their highest levels in five weeks on Tuesday, as threats from former US President Donald Trump regarding tariffs on Russian crude and potential action against Iran eased concerns about the impact of trade wars on global growth.
    • Quote: “يف حني أن العقوبات الأكرث صرامة على إيران وفنزويا وروسيا قد تقيد الإمدادات العاملية، فمن املرجح أن تضعف الرسوم الجمركية الأمريكية الطلب العاملي على الطاقة وتبطئ النمو االقتصادي، مما يؤثر بدوره يف النفط على الطلب على النفط.” (Page 16, quoting Oil Heapalby, analyst at ANZ Bank)
    • Key Concern: Analysts highlight the potential for US tariffs to weaken global energy demand and slow economic growth, offsetting the impact of supply constraints from sanctions.
    • Non-Oil Exports: A report from the General Authority for Statistics on non-oil exports during the fourth quarter of 2024 is discussed.
    • Key Observation: A significant decrease in total non-oil re-exports (reaching 35.2%) and a decrease in total imports (55.3%) were noted compared to previous periods.
    • Key Idea: The export process is complex and influenced by many factors beyond government control. There is a need to work on overcoming obstacles, enhancing the internal export environment, and proactively adapting to changes through scenario planning.
    • Key Data: The UAE remained the top destination for non-oil exports (26.7%), followed by India (9.4%) and China (8.1%) as of November 2024.
    • Key Recommendation: Diversifying export destinations, particularly focusing on African (excluding Arab and Islamic), Latin American, and non-EU European countries, is crucial.
    • Investment Opportunities and Challenges: A report from HSBC Private Banking highlights key priorities for the second quarter of 2025, including adapting to AI and global profit growth driven by clean energy and AI service providers.
    • Key Idea: Diversifying across asset classes, economic sectors, geographical regions, and currencies offers opportunities to enhance risk-adjusted returns.
    • Key Observation: While the global economy faces challenges, it remains resilient due to increased government and corporate spending on innovation and rising productivity.
    • Regional Strength: The economies of Saudi Arabia and the UAE are seen as strong compared to others in the Gulf Cooperation Council, supported by sovereign wealth, robust reserves, and ongoing structural reforms and infrastructure projects.
    • Challenge: The increasing commercial frictions and the rise of AI-based innovations pose significant challenges to markets, requiring wealthy clients to adapt.
    • Capital Market Authority (CMA) Initiative: The CMA has invited public feedback on a draft to improve the governance of Special Purpose Entities (SPEs) and streamline their procedures.
    • Key Objective: To enhance the attractiveness of SPEs as investment vehicles for specific purposes and facilitate the issuance of debt instruments (Sukuk and bonds).
    • Expected Outcome: The project aims to develop the Sukuk and debt instrument market, diversify issuances, and boost liquidity.
    • Significant Growth: The number of established SPEs saw a significant increase from 464 between 2018 and 2023 to 945 existing entities.
    • Listing of Umm Al-Qura for Development and Construction Company: The IPO of Umm Al-Qura is highlighted as a significant event reflecting the maturity of the Saudi capital market and the depth of the Kingdom’s investment vision.
    • Key Fact: The IPO offered 130,786,142 ordinary shares, reflecting strong investor confidence.
    • Project Details: The proceeds will primarily fund the “Masar” project in Makkah, a large-scale development including 54 residential projects, 66 hotel towers, 4 commercial projects, and a 3.6 km pedestrian walkway, aiming to enhance the experience for visitors and residents.
    • Positive Indicator: The fully subscribed and oversubscribed IPO demonstrates the solid faith investors have in the Saudi economy and the attractiveness of available investment opportunities.
    • Shell’s Asset Sale in Singapore and Market Dynamics: Shell completed the sale of its Bukom and Jurong Island refining and petrochemical assets in Singapore to Chandra Asri Petrochemical (CAP).
    • Market Shift: The new owners have already begun purchasing raw materials, indicating a change in market dynamics.
    • Supply Chain Adjustments: Chandra Asri has undertaken several spot purchases of naphtha and assumed responsibility for supplying raw petrochemical materials to Astarte for Singapore shipments.
    • Unusual Trade Flows: Data shows rare shipments of Canadian crude oil heading to Singapore under Glencore’s trading operations, coinciding with Shell’s divestment.
    • Caspian Pipeline Consortium (CPC) Disruptions: Russia ordered the closure of three mooring stations of the CPC, which handles about 1% of global oil supplies from Kazakhstan’s Kashagan field.
    • Reason: Ostensibly due to surprise inspections by the Russian transport regulatory authority.
    • Potential Impact: The halt could lead to a significant decrease in oil exports if it lasts more than a week.
    • Geopolitical Context: The move came shortly after reports of former US President Trump being dissatisfied with Russia and the progress of peace talks with Ukraine, along with his tariff threats. Russia also cited previous technical issues, storms, and a Ukrainian drone attack affecting the pipeline’s operations.
    • European Commission and Chemical Industry: The European Commission is working to help the chemical sector manage high energy prices, modernization costs, and the transition.
    • Key Objective: To enhance the sector’s competitiveness.
    • Urgent Need: Updating steam cracking units older than 40 years is crucial due to their environmental inefficiency and weak performance.
    • Call for Action: Eight European countries urged measures to support chemical production amid rising costs and competition. They emphasized the importance of a proposed “Critical Raw Materials Act” in supporting the development and decarbonization of chemical plants and promoting alternative carbon sources.
    • Plant Closures: Several chemical plants and cracking units in Europe have closed or are operating at reduced rates due to high costs and fading competitive advantages. Similar challenges and closures are affecting polyolefin plants in Southeast Asia due to weak profit margins, oversupply, and high naphtha prices. The cost of producing ethylene from naphtha in Asia is significantly higher than in Saudi Arabia and the US, putting pressure on Asian producers.

    III. Regional and International Affairs:

    • Israeli Strike in Southern Beirut: An Israeli airstrike on a southern suburb of Beirut killed a Hezbollah official, leading to condemnation from Lebanese officials and raising concerns about escalating tensions.
    • Casualty: A “field official” ( مسؤول حزبي) in Hezbollah was killed.
    • Lebanese Reaction: President Michel Aoun described the strike as a dangerous warning about hostile intentions against Lebanon and a blatant violation of UN Resolution 1701. Parliament Speaker Nabih Berri, a Hezbollah ally, called it an attempt to sabotage the ceasefire agreement. Hezbollah MP Ibrahim Al-Moussawi also condemned the attack.
    • Hezbollah’s Response: While condemning the attack, Hezbollah’s immediate response was cautious, with some suggesting the group might be overestimating its strength.
    • UN Resolution 1701: The strike is seen as a clear breach of the resolution that ended the 2006 war between Israel and Hezbollah.
    • Syrian Political Developments: The United States considered the formation of a new Syrian government a positive step but stated that it would not ease restrictions until progress is made on priorities, including combating “terrorism.”
    • Gaza Humanitarian Crisis: The situation in Gaza is dire, with accusations against Israel of deliberately causing mass starvation by closing crossings and preventing the entry of aid.
    • Accusations: The Palestinian side claims Israel has prevented the entry of 18,600 truckloads of aid and essential fuel since the start of the war, bombed over 60 food kitchens and distribution centers, and targeted bakeries.
    • Bakery Closures: The head of the bakery owners’ association in Gaza reported that all bakeries have closed due to the depletion of flour and fuel as a result of the ongoing siege and prevention of aid.
    • International Aid Obstruction: The UN humanitarian affairs agency reported that Israeli authorities rejected 40 out of 49 requests to coordinate aid deliveries to Gaza in March.
    • Impact on Children: Over 100 children are reportedly killed or injured daily in Gaza, with more than 15,000 killed and over 34,000 injured in recent months. UNICEF emphasized the urgent need for humanitarian access and protection for children.

    IV. Social and Cultural Snippets:

    • The “Ardah” as a Leading Folk Art: The “Ardah” (traditional Saudi sword dance) is highlighted as a prominent and captivating folk art, enjoying significant popularity.
    • Role of Poetry in Connecting Generations: Poetry is presented as a powerful medium for sharing feelings, ideas, and cultural heritage across generations, with traditional poems being passed down and cherished.
    • Saudi Arabia’s Commitment to Pilgrims and Visitors: The Kingdom’s efforts to ensure the safety, security, and tranquility of pilgrims, visitors, and residents are emphasized, rooted in Islamic values and the teachings of the Quran. The dedication of Saudi security personnel is commended.
    • The Significance of Eid Celebrations: Eid is described as embodying profound human values, fostering social cohesion, preserving heritage, and emphasizing the balance between work and rest. The spiritual and social dimensions of Eid Al-Fitr and Eid Al-Adha are highlighted, focusing on gratitude, compassion, and strengthening family ties.
    • Importance of Laws and Regulations: The necessity of adhering to laws and regulations for the security, justice, and equality of societies is stressed. Violating these norms leads to chaos and hinders progress and the protection of individual and societal rights.
    • Khaled Abdulrahman’s Eid Concert: The renowned artist Khaled Abdulrahman held a successful Eid concert in Qassim, part of a series organized by the General Entertainment Authority (GEA).
    • Youssef Farah’s Musical Ventures: Actress Youssef Farah is pursuing a singing career alongside her acting, having released several songs in 2024 and preparing new releases for Eid Al-Fitr.

    V. Sports Highlights:

    • Saudi National Futsal Team Camp in Vietnam: The national futsal team is holding an overseas training camp in Vietnam in preparation for the Asian Cup qualifiers.
    • Saudi National Football Team’s World Cup Qualifiers: The national football team faces crucial matches in the World Cup qualifiers, needing to secure points to improve their chances of direct qualification or reaching the playoffs.
    • Al-Ittihad Reaches King’s Cup Final: Al-Ittihad qualified for the King’s Cup final after an exciting victory over Al-Shabab.
    • Al-Qadsiah and Al-Raed Compete for Second Final Spot: Al-Qadsiah and Al-Raed are set to play in the second King’s Cup semi-final match.
    • Al-Faisaly vs. Al-Batin and Jeddah vs. Al-Hazm League Matches: Previews of upcoming league matches are provided, highlighting team standings and key players.
    • Mosaab Al-Juwair: A Rising Football Talent: Young player Mosaab Al-Juwair is highlighted as a promising talent in Saudi football, showcasing impressive performances for Al-Shabab and earning recognition as a key player for the national team. He has received multiple “Roshen League Young Player of the Month” awards and is attracting interest from both local and European clubs.
    • Japan’s Football Development: From 2005 Dream to 2050 Vision: The remarkable progress of Japanese football over the years is discussed, tracing its roots to a long-term vision set in 2005 to win the World Cup by 2050. Factors contributing to this development include investing in youth academies, focusing on tactical and intellectual development, strong school football programs (boosted by the “Captain Tsubasa” manga), and sending talents to Brazil (though this strategy was later revised). Japan’s hosting of the 2002 World Cup marked a significant milestone.
    • Al-Riyadh Club’s Role in Developing Young Talents: Despite limited financial resources, Al-Riyadh Club plays a vital role in developing young football talents through its academy and youth teams, providing a platform for promising players and employing a comprehensive training and scouting strategy.

    VI. Miscellaneous:

    • “Zakat, Tax and Customs Authority” Inspection Campaigns: The authority conducted over 12,000 inspection visits in March 2025 across various commercial sectors to ensure compliance with tax regulations and combat violations like not issuing e-invoices. Consumers are encouraged to report violations.
    • Al-Aan Heritage Palace in Najran: The palace, known for its unique architecture, attracts visitors during Eid Al-Fitr, showcasing the region’s heritage and traditional crafts.
    • Drowning Prevention Awareness: The Civil Defense in the Eastern Province issued warnings about drowning hazards during Eid gatherings, particularly around swimming pools, emphasizing the importance of supervision and safety measures.
    • Psychological Well-being during Eid: A consultant psychiatrist emphasizes the psychological benefits of Eid celebrations, including breaking routine, fostering social connection, spreading optimism, and promoting a sense of belonging and self-esteem.

    This briefing provides a comprehensive overview of the key information presented in the “Al Riyadh” newspaper excerpts, highlighting domestic developments, economic trends, regional issues, social observations, and sports news as of April 2, 2025.

    FAQ: Key Themes from the Saudi Arabian Newspaper “Al Riyadh” (April 2, 2025)

    1. What strategic importance does Saudi Arabia place on protecting children in cyberspace? Saudi Arabia considers protecting children in cyberspace a strategic investment, recognizing its role in enhancing the quality of life and fostering development. This commitment was highlighted during the launch of the first global cyber security summit for child protection, an initiative by Crown Prince Mohammed bin Salman aimed at unifying international efforts to address threats faced by children online and build capabilities in this area.

    2. What are some recent advancements and initiatives by Saudi Arabia in the space sector? Saudi Arabia has made significant strides in its space program. This includes the launch of several satellites like “Shaheen” for telecommunications and internet services, the development of spacecraft such as “Najm” for space exploration, and the launch of the Saudi Space Program aimed at promoting scientific research and technological advancement in space. A notable achievement is the launch of the first Saudi research mission to study and analyze the space microbiome in low Earth orbit, marking a significant step in space exploration and research.

    3. What is the outlook for the oil market according to the article, particularly concerning OPEC+ decisions and global economic factors? The oil market is closely watching the upcoming OPEC+ ministerial committee meeting in April. Sources indicate that OPEC+ is expected to proceed with planned production increases. Oil prices stabilized recently, despite earlier concerns related to potential US tariffs on Russian crude and tensions with Iran. However, analysts suggest that while stricter sanctions on Iran, Venezuela, and Russia could limit global supply, US tariffs might weaken global energy demand and slow economic growth, making a clear market direction uncertain.

    4. How is Saudi Arabia demonstrating its commitment to combating climate change and promoting environmental sustainability? Saudi Arabia views environmental protection as a fundamental part of its future plans, exemplified by the Saudi Green Initiative. This initiative aims to create a more sustainable future by focusing on increasing reliance on renewable energy sources like solar and wind power, protecting 30% of the Kingdom’s land as natural reserves to conserve biodiversity, and planting 10 billion trees across the country in the coming decades to combat desertification and improve air quality. The Kingdom is actively working towards achieving its goal of reducing carbon emissions by 60% by 2030.

    5. How has the celebration of Eid Al-Fitr evolved in Saudi Arabia with the rise of social media and the internet? While traditional in-person family visits were the primary way of exchanging Eid greetings in the past, the rise of social media and the internet has led to a significant shift. Now, phone calls, voice messages, short text messages on platforms like Twitter, and posts on Facebook and Instagram have become common and efficient ways to convey festive wishes, especially for those living far from family or with busy schedules.

    6. What are the key challenges and opportunities identified in the non-oil export sector in Saudi Arabia? Saudi Arabia’s non-oil exports saw a decrease in the fourth quarter of 2024, along with a decline in re-exports and air freight exports. The primary destination for non-oil exports remains the UAE, followed by India and China. While challenges such as external controls and existing technical regulations in potential markets exist, there is significant room for improvement by enhancing the internal export environment, adapting to changing conditions through forecasting, and diversifying export destinations to regions like Africa (excluding Arab and Islamic nations), Latin America, and non-EU European countries.

    7. What are the primary objectives of Saudi Arabia’s project to enhance the attractiveness of Special Purpose Entities (SPEs) through improved governance? The Saudi Capital Market Authority (CMA) is undertaking a project to improve the governance of SPEs to enhance their appeal for investment purposes and facilitate their procedures for issuing debt instruments and sukuk. The project aims to strengthen the legal standing of SPEs, develop the debt and sukuk market, diversify issuances, increase investment opportunities, and boost liquidity. Key aspects of the reform include adding requirements for the personal guardian of the SPE and developing provisions for their dismissal, as well as reviewing the independence requirements for the SPE’s board members from the founder and sponsor.

    8. What does the successful IPO of Umm Al-Qura for Development and Construction signify for the Saudi Arabian economy and real estate market? The IPO of Umm Al-Qura for Development and Construction is seen as a significant milestone, reflecting the maturity of the Saudi financial market and the depth of the Kingdom’s investment vision. This IPO, aimed at funding the “Masar Destination” project in Makkah, which includes hotels, residential towers, and commercial spaces, signifies a redefinition of real estate development in the holy city. The strong investor confidence and full subscription of the IPO underscore the resilience of the Saudi economy and the attractiveness of available investment opportunities, aligning with the goals of Vision 2030 to develop promising sectors and accommodate major projects.

    OPEC+, Production, and Global Oil Market Dynamics

    The sources discuss OPEC+ oil production in the context of several factors.

    Kazakhstan’s Production and OPEC+ Agreement: According to the sources, Kazakhstan has exceeded its oil production quotas as part of an agreement between OPEC+ producers and their allies, including Russia.

    Russia’s Production Challenges: The sources indicate that Russia, also part of OPEC+, is facing difficulties in persuading companies operating its oil fields to reduce production.

    Potential Impact of US Tariffs on Russian Oil: There is mention of threats by the US to impose tariffs on Russian oil, the second-largest global oil exporter, which could disrupt supplies and potentially affect global oil markets.

    OPEC+ Supply and Global Demand: One source states that OPEC+ is increasing its supply. Simultaneously, there are concerns that a slowdown in global economic growth and specifically in China could lead to a decrease in the demand for fuel, potentially offsetting any reduction in supply caused by the tariff threats.

    Caspian Pipeline Consortium (CPC) Disruption: The sources report a temporary disruption in the flow of oil through the Caspian Pipeline Consortium (CPC), which exports approximately 1% of global oil supplies. This occurred after inspection operations by the Russian transport regulator. The disruption could lead to a further decrease in oil exports if it persists for more than a week.

    Kazakhstan’s Response to CPC Disruption: Due to the reduced volume of oil flowing through the CPC pipeline, Kazakhstan is reportedly considering reducing its oil production.

    Gaza: Humanitarian Crisis and Allegations of Genocide

    The sources indicate a severe humanitarian crisis in Gaza. According to the governmental media office in Gaza, the sector is dying gradually due to starvation and collective extermination carried out by the Israeli occupation against civilian life, resulting in the killing of children and women.

    The Israeli occupation is accused of exceeding all limits in its actions against unarmed civilians. The media office calls these actions a “crime of genocide” and demands urgent international intervention to stop it.

    The sources also highlight the following:

    • A call for the International community to take immediate action to stop the “crime of genocide”.
    • A demand to hold Israeli war criminals and those responsible for the aggression accountable before international courts.
    • A request for an immediate international and independent investigation into the various war crimes committed by the occupation against the Palestinian people.
    • A call to exert pressure and compel the occupation to end the unjust siege on Gaza.
    • A necessity for allowing the entry of humanitarian aid, fuel, and medical supplies without any restrictions, especially for wounded civilians and newborns.

    Protecting Children in Cyberspace: An International Responsibility

    The sources do not contain information about a “Cyber security summit”. However, one of the main themes discussed in the provided excerpts is the protection of children in cyberspace.

    The Kingdom of Saudi Arabia considers protecting children in the cyber environment a strategic investment for a more secure and sustainable future. The statement highlights that this issue is not merely a technological challenge but a crucial investment.

    The sources emphasize that many nations, particularly those facing developmental challenges, still lack the necessary digital infrastructure and resources to protect children from online risks. This necessitates strengthening capacity building and bridging these gaps through international support.

    The sources also include a statement calling for unifying international efforts and partnerships between governments and the private sector to develop practical and sustainable solutions for child protection. It further urges the United Nations High Commissioner for Human Rights to provide technical assistance to countries in need, including developing national legislation, training those working in law enforcement, and establishing safe reporting mechanisms.

    The excerpts conclude by affirming that protecting children in cyberspace is a shared international responsibility that requires ensuring the digital world is a safe environment where children’s rights are respected and their dignity is guaranteed. This statement aligns with the Kingdom’s ongoing efforts and concern for protecting and enhancing the safety and well-being of children in the digital environment.

    Therefore, while the sources do not address a specific “Cyber security summit,” they extensively discuss the critical cybersecurity issue of child protection in the digital realm and the necessary collaborative efforts to address it.

    Jazan Region Eid al-Fitr Celebrations and Governor’s Address

    The sources mention celebrations in the Jazan region specifically in the context of Eid al-Fitr. His Royal Highness Prince Mohammed bin Nasser bin Abdulaziz, the Governor of the Jazan region, received well-wishers on the occasion of Eid al-Fitr.

    The well-wishers included deputies of governorates, heads of centers, tribal chiefs, and a group of citizens who came to greet His Royal Highness and offer their congratulations on the arrival of Eid al-Fitr.

    During this reception, the Prince emphasized the attention and care provided by the wise leadership to all citizens and residents of the Jazan region, with the aim of ensuring their comfort and prosperity. He also underscored the significant role of citizens in the national development and progress of the country, as well as the importance of preserving security, safety, capabilities, achievements, and the values upon which the Kingdom was founded – the principles of Islam and Islamic law. The Governor concluded by praying to God for the continued security and well-being of the nation.

    Therefore, the celebration in the Jazan region, as highlighted in the sources, centered around the traditional greetings and expressions of goodwill on the occasion of Eid al-Fitr, along with the Governor’s address emphasizing national values and the leadership’s commitment to the well-being of the region’s inhabitants.

    Saudi Non-Oil Export Diversification: Destinations and Challenges

    The sources discuss Saudi Arabia’s non-oil exports and the importance of diversifying their destinations. It is stated that without a doubt, non-oil exports are crucial, and the primary destination for these exports was the UAE, followed by India and then China, as of November 2024.

    The source emphasizes the need to increase focus on African countries (excluding Arab and Islamic ones), Latin America, and non-EU European Union countries. Despite the anticipated potential, the Kingdom’s exports to these nations still face challenges due to technical and regulatory factors, which hinder their access compared to other countries within the EU.

    The source does not explicitly mention a decline in non-oil exports. Instead, it focuses on the process of exporting as a highly complex and dynamic operation influenced by numerous factors, many of which are beyond the control of governments. It suggests that there is significant room for improvement by working to overcome all obstacles and enhancing the internal environment and sustainability of exports. This improvement should be proactive, adapting to changing conditions and variables through scenario forecasting.

    In summary, while the source highlights the significance of non-oil exports and the need for diversification along with the challenges in accessing certain markets, it does not provide information indicating a decline in these exports.

    Main Headings

    • المملكة: حماية الأطفال في الفضاء السيبراني استثمار استراتيجي Kingdom: Protecting children in cyberspace is a strategic investment
    • المملكة ترحب بتوقيع طاجيكستان وقرغيزستان وأوزباكستان معاهدة الحدود المشتركة The Kingdom welcomes the signing of the Joint Border Treaty by Tajikistan, Kyrgyzstan, and Uzbekistan.
    • مجلس التنفيذيين اللبنانيين» يعزز العلاقات مع المملكة The Lebanese Executive Council strengthens relations with the Kingdom.
    • المدى البصري ، نوال الجبر Visual range, Nawal Al-Jaber
    • سوق النفط يترقب اجتماع «وزارية أوبك..»+ السبت The oil market awaits the OPEC+ ministerial meeting on Saturday.
    • السعودية الخضراء..» حضور عالمي لمكافحة التغير المناخي Green Saudi Arabia: A Global Presence to Combat Climate Change
    • إســــــرائــــــيــــــل) تـــقـــتـــل مـــــــســـــــ ً ؤولا فــــــي حــــــــزب الـــلـــه ، غـزة تمـوت تدريجيـاً (Israel) kills a Hezbollah official, Gaza is gradually dying
    • أميـر جــازان يستقـبل المهنئيــن بالــعيـد Emir of Jazan receives Eid well-wishers
    • أمير تبوك يلتقي أهالي تيماء Emir of Tabuk meets with Tayma residents
    • محافظ الدرعية يرعى حفل العرضة السعودية The Governor of Diriyah sponsors the Saudi Ardah ceremony.
    • أهالي وادي الدواسر يحتفلون بالعيد Wadi Ad-Dawasir residents celebrate Eid
    • العرضة والسامري في عيد عفيف Al-Ardah and Al-Samri on Eid Al-Afif
    • القنصلية في سيدني تنظم فعاليات وأنشطة رياضية للطلاب The Consulate in Sydney organizes sports events and activities for students.
    • السفارة السعودية في أميركا تحتفل بعيد الفطر The Saudi Embassy in the United States celebrates Eid al-Fitr.

    By Amjad Izhar
    Contact: amjad.izhar@gmail.com
    https://amjadizhar.blog

  • Excel Mastery for Data Analysis in 2025 Comprehensive Guide AI Powered Copilot

    Excel Mastery for Data Analysis in 2025 Comprehensive Guide AI Powered Copilot

    These learning materials offer a comprehensive guide to mastering Microsoft Excel, starting with fundamental interface elements like the ribbon and cells. They progress to essential functions such as SUM and IF, alongside data manipulation techniques including sorting, filtering, and removing duplicates. The resources also cover data visualization through various charts and pivot tables, as well as more advanced tools like Power Query and the use of the AI-powered Copilot. Furthermore, they explore the role of Excel in data analysis, covering statistical functions and the creation of interactive dashboards. Finally, the materials touch upon career paths in data analysis, emphasizing required skills and the importance of projects and portfolios. The practical demonstrations and step-by-step instructions aim to equip users with both basic and intermediate Excel proficiencies.

    Excel Proficiency Study Guide

    Quiz

    1. Explain the primary difference between the COUNT and COUNTA functions in Excel. Provide a brief example illustrating this difference.
    • The COUNT function only counts cells containing numerical values within a specified range, while COUNTA counts all non-empty cells, including those with numbers, text, dates, and logical values. For example, if range A1:A3 contains “1”, “apple”, and a blank cell, COUNT would return 1, and COUNTA would return 2.
    1. Describe how the COUNTIF function is used in Excel. Give a simple scenario where using COUNTIF would be beneficial.
    • The COUNTIF function counts the number of cells within a specified range that meet a given criteria. A beneficial scenario would be counting how many students in a class scored above a certain grade on an exam by specifying the score threshold as the criteria within the range of student scores.
    1. What are the steps involved in formatting dates in Excel? Mention at least two different date formats one might apply.
    • To format dates, select the cells containing the dates, right-click, and choose “Format Cells.” In the “Number” tab, select “Date” from the category list. Then, choose the desired format from the available options or create a custom format. Two different date formats are “MM/DD/YYYY” and “DD-MMM-YY”.
    1. Explain the purpose of the DATEDIF function in Excel. What are the common units used to calculate the difference between two dates with this function?
    • The DATEDIF function calculates the difference between two dates based on a specified unit. Common units used include “Y” for years, “M” for months, and “D” for days. It’s often used to calculate age or the duration between events.
    1. Describe the ROUND function in Excel and its basic syntax. Provide an example of how it can be used to simplify decimal values.
    • The ROUND function rounds a number to a specified number of digits. The basic syntax is =ROUND(number, num_digits). For example, =ROUND(80.66, 1) would round the number 80.66 to one decimal place, resulting in 80.7.
    1. Briefly explain the concept of standard deviation and variance in the context of Excel and data analysis. How are these measures helpful?
    • Standard deviation measures the dispersion or spread of data points around the mean, while variance is the average of the squared differences from the mean. These measures are helpful in understanding the variability within a dataset; a higher standard deviation or variance indicates greater spread.
    1. How does the MATCH function work in Excel? What are the key arguments required for this function?
    • The MATCH function searches for a specified item in a range of cells and then returns the relative position of that item in the range. The key arguments are lookup_value (the item to search for), lookup_array (the range to search within), and match_type (specifying exact or approximate match).
    1. Outline the purpose and basic steps for using the SUMIFS function in Excel. How does it differ from the SUMIF function?
    • The SUMIFS function calculates the sum of cells in a range that meet multiple criteria. The basic steps involve selecting the sum range, then specifying pairs of criteria ranges and their corresponding criteria. Unlike SUMIF, which allows only one condition, SUMIFS can handle multiple conditions.
    1. What is a macro in Excel, and how can it be used to count colored cells based on manual formatting?
    • A macro in Excel is a recorded or programmed sequence of actions that can automate tasks. A macro to count manually colored cells would involve VBA code that iterates through a specified range, checks the interior color of each cell against a given color code, and increments a counter for each match.
    1. Explain the core idea behind time series data and mention its primary components as discussed in the source material.
    • Time series data is a sequence of data points recorded over specific intervals of time, making it time-dependent. Its primary components are trend (the overall direction of the data), seasonality (periodic fluctuations), cyclicity (longer-term fluctuations), and irregularity (random variations).

    Essay Format Questions

    1. Discuss the importance of data validation in Excel. Describe different data validation techniques and explain how they contribute to data accuracy and integrity.
    2. Compare and contrast the use of formulas and functions in Excel. Provide examples of when you might use each and explain the order of operations Excel follows when evaluating formulas.
    3. Explain the functionality and benefits of using the VLOOKUP function in Excel. Describe a scenario where VLOOKUP would be particularly useful and discuss its limitations compared to other lookup functions.
    4. Discuss the capabilities of Excel for data analysis and visualization. Describe how features like pivot tables, charts, and the Data Analysis Toolpak can be used to gain insights from data, referencing specific examples from the source material where applicable.
    5. Explain the concept of time series forecasting and the ARIMA model as introduced in the source material. Discuss the key components of time series data and the parameters of the ARIMA model (p, d, q), and the role of ACF and PACF in model selection.

    Glossary of Key Terms

    • COUNT Function: An Excel function used to count the number of cells within a specified range that contain numerical values.
    • COUNTA Function: An Excel function used to count the number of non-empty cells within a specified range, regardless of the data type.
    • COUNTIF Function: An Excel function used to count the number of cells within a specified range that meet a specific criterion.
    • Date Formatting: The process of changing the way dates are displayed in Excel, including the order of day, month, year, and the use of separators.
    • DATEDIF Function: An Excel function that calculates the difference between two dates in specified units like years, months, or days.
    • ROUND Function: An Excel function that rounds a number to a specified number of digits.
    • Standard Deviation: A measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, while a high standard deviation indicates that the values are spread out over a wider range.
    • Variance: A measure of how far a set of numbers is spread out from their average value. It is calculated as the average of the squared differences from the mean.
    • MATCH Function: An Excel function that searches for a specified item in a range of cells and returns the relative position of that item in the range.
    • SUMIFS Function: An Excel function that calculates the sum of cells in a range that meet multiple specified criteria.
    • Macro: A recorded or programmed sequence of actions in Excel that can automate repetitive tasks.
    • Time Series Data: A sequence of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time.
    • Trend (Time Series): The long-term movement in a time series, indicating the general direction (upward or downward) of the data over a sustained period.
    • Seasonality (Time Series): Regular and predictable fluctuations in a time series that occur within a year, often repeating annually.
    • Cyclicity (Time Series): Longer-term fluctuations in a time series that occur over periods longer than a year, often related to economic or business cycles.
    • Irregularity/Random Component (Time Series): Unpredictable and short-term fluctuations in a time series that do not follow a regular pattern.
    • ARIMA Model: An acronym for AutoRegressive Integrated Moving Average, a class of statistical models for analyzing and forecasting time series data.
    • Autoregressive (AR) Term: In an ARIMA model, the component that uses the dependency between an observation and a number of lagged observations.
    • Integrated (I) Term: In an ARIMA model, the component that represents the number of times the raw observations are differenced to make the time series stationary.
    • Moving Average (MA) Term: In an ARIMA model, the component that uses the dependency between an observation and a residual error from a moving average model applied to lagged observations.
    • Stationary Data: A time series whose statistical properties such as mean, variance, autocorrelation, etc. are all constant over time.
    • Autocorrelation Function (ACF): A function that shows the correlation of a time series with its own past values.
    • Partial Autocorrelation Function (PACF): A function that shows the correlation of a time series with its own lagged values, controlling for the correlations at intermediate lags.

    Briefing Document: Review of Excel Features and Functions

    This document provides a detailed review of the main themes and important ideas or facts presented in the provided excerpts from an Excel course. The sources cover a range of fundamental and advanced Excel functionalities, including data manipulation, analysis, visualization, and automation.

    1. Basic Functions for Data Aggregation and Analysis:

    • COUNT Function: This function is used to count the number of cells within a specified range that contain numbers. It is highlighted as crucial for quickly assessing numerical density in mixed data sets.
    • Example: =COUNT(J2:J5) applied to cells containing 7, 89, 0, 100 would return 4.
    • Quote: “the count function in Excel is used to specifically count the number of cells that contain numbers in a Range”
    • COUNTA Function: Unlike the COUNT function, COUNTA counts the number of non-empty cells in a range. This includes numbers, text, dates, and logical values. It is useful for determining the size of data sets regardless of the data type.
    • Difference from COUNT: COUNT only counts numerical values and dates (excluding blanks), while COUNTA counts any non-empty cell, including text.
    • Example: =COUNTA(K2:K5) applied to cells containing 56, a blank cell, 98, and 56 would return 3. It would also return 3 if the cells contained “Apple”, a blank cell, “Banana”, and “Orange”.
    • Quote: “this count a function in Excel counts the number of cells in a Range that are not empty. […] the count function generally used to count a range of cells containing numbers or dates excluding blank whereas this count a function will count everything including numbers dates text or a range containing a mixture of these items but does not count blank cells”
    • COUNTIF Function: This function allows counting cells within a range that meet a specified criteria. This is valuable for conditional counting.
    • (Further details on COUNTIF criteria are expected in subsequent parts of the full course.)
    • Quote: “this counter function is used to count the number of cells in a Range that meet a specified criteria.”
    • (Other basic functions mentioned but not detailed in this excerpt include CONCATENATE, TRIM, MAX, MIN, AVERAGE, IF, and SUM.)

    2. Data Formatting:

    • Date Formatting: Excel offers flexibility in how dates are displayed. Users can change the format by selecting cells, right-clicking, choosing “Format Cells,” and navigating to the “Date” category or using “Custom” formatting.
    • Location-based formatting: Dates can be displayed according to different regional conventions (e.g., US format: YYYY-MM-DD, India format: DD-MM-YYYY).
    • Custom formatting: Users can specify which parts of the date to display (e.g., only month and year: “MMMM YYYY”).
    • Inclusion of time: Date formats can also include time, down to seconds or milliseconds.
    • Examples: Switching from DD-MM-YYYY to MM-DD-YYYY by selecting “English (United States)”. Displaying only month and year by using a custom format like “MMMM YY”. Displaying date and time including seconds using a format like “DD-MM-YYYY HH:MM:SS”.
    • Quote: “you might want to also change your dates based on the location so right now we are in India and imagine if you wanted to you know change something based on us or if you if you’re having your client in us and he wants the dates in US format it you can also change that […] you might want to also change your dates based on the location”

    3. Calculating Age from Date of Birth:

    • DATEDIF Function (or DATE DIFF): This function is used to calculate the difference between two dates in various units (years, months, days).
    • Syntax: =DATEDIF(start_date, end_date, unit)
    • Units: “Y” for years, “M” for months, “D” for days.
    • Example: =DATEDIF(“2010/01/05”, “2022/01/01”, “Y”) would return the difference in years.
    • Quote: “all you have to use is the dated IF function or if you also call it as date diff function based on your choice”

    4. Rounding Numbers:

    • ROUND Function: This function rounds a number to a specified number of digits.
    • Syntax: =ROUND(number, num_digits)
    • num_digits: Specifies the number of digits to which the number should be rounded. Positive for decimal places, zero for the nearest integer, negative for rounding to the left of the decimal point.
    • Example (implicitly using ROUNDUP): =ROUNDUP(M3, 1) applied to 80.66 would round up to 80.7.
    • Quote: “to actually perform the roundoff uh formula we do have a pre method for that for that you just have to type in equals to round and there you go we have round up and round down so both of them perform the same operation” (Note: The example uses ROUNDUP, but the general concept of rounding is introduced)

    5. Understanding and Calculating Standard Deviation and Variance:

    • Standard Deviation: Defined as the calculated square root of variance. It measures the dispersion or spread of data points around the mean. A higher standard deviation indicates greater variability.
    • Prerequisites for manual calculation: Mean, variance, deviation (difference between observed value and expected/mean value), and squared deviation.
    1. Steps for manual calculation:Calculate the mean of the data set.
    2. Calculate the deviation of each data point from the mean.
    3. Square each deviation.
    4. Calculate the variance (sum of squared deviations divided by the total number of values minus 1 for sample standard deviation).
    5. Calculate the standard deviation (square root of the variance).
    • Excel Function: Excel has built-in functions to directly calculate standard deviation (e.g., STDEV.S for sample, STDEV.P for population).
    • Quote: “standard deviation is a calculated square of variance”
    • Variance: A measure of variability, calculated as the average of the squared deviations from the mean. It indicates how far data points are spread out from the average, in squared units.
    • Quote: “variance is a measure of variability it is calculated by taking the average of squared deviation from the mean”
    • Deviation: The difference between an observed value and the expected value (often the mean). It represents the distance from the center point.
    • Quote: “the deviation is a measure that is used to find the difference between the observed value and the expected value of a variable in simple terms deviation is the distance from the center point”

    6. Using the MATCH Function:

    • Purpose: The MATCH function is used to find the position (index) of a specified item within a range of cells.
    • Syntax: =MATCH(lookup_value, lookup_array, [match_type])
    • lookup_value: The value you want to find.
    • lookup_array: The range where you want to search.
    • match_type: (Optional) 0 for exact match, 1 for less than, -1 for greater than.
    • Example: =MATCH(“Marketing”, A2:A11, 0) would return 4 if “Marketing” is the fourth item in the range A2:A11.
    • Use Case: Helpful for quickly identifying the position of a specific element in a list.
    • Quote: “our main idea is to find out the index of these particular elements for example if you wanted to find the index of the element marketing then how could you do it so here let’s try to use the match function in Excel”

    7. Creating Custom Functions with Macros (VBA):

    • Purpose: Macros allow users to automate tasks and create custom functions in Excel using VBA (Visual Basic for Applications).
    • Example: COUNT COLORED CELLS: The excerpt describes a macro function called count colored cells designed to count the number of manually colored cells within a specified range that match the interior color of a selected “current cell.”
    • Logic: The macro iterates through each cell in the given range. It compares the interior color of each cell to the interior color of the “current cell.” If the colors match, a counter variable is incremented.
    • Usage: =countcoloredcells(current_cell, cell_range)
    • Limitation: This specific macro works only for manually colored cells and will not recognize colors applied through conditional formatting.
    • Quote: “this particular macro will work only for manually colored cells so there are situations where we have used conditional formatting to color a single cell […] if we try to use that particular green color to be counted by the color count sales function no it will not happen”

    8. Identifying and Removing Duplicate Rows:

    • Excel Functionality (implied): While not explicitly detailing the steps, the excerpt mentions the possibility of duplicate rows in a student database and implies that Excel has methods to identify and remove them.
    • (Standard Excel features for this include “Remove Duplicates” under the “Data” tab, which allows users to select specific columns to consider when identifying duplicates.)
    • Quote: “there might be possibilities about the duplication of class because all the students all the 10 students are in same class so there might be a duplication but we are not looking for such kind of duplications right we are looking for the duplication of the entire row”

    9. Using the SUMIFS Function for Conditional Summation:

    • Purpose: SUMIFS allows summing values in a range based on multiple criteria.
    • Syntax: =SUMIFS(sum_range, criteria_range1, criteria1, [criteria_range2, criteria2], …)
    • sum_range: The range of cells to sum.
    • criteria_range(n): The range of cells to evaluate the criteria against.
    • criteria(n): The condition that must be met.
    • Prerequisite: Converting the data range into an Excel Table using Ctrl+T is recommended for easier referencing of columns.
    • Example: =SUMIFS(Table1[Sales], Table1[Region], “West”) would sum the values in the “Sales” column of “Table1” where the corresponding value in the “Region” column is “West”.
    • Multiple Criteria: SUMIFS can accommodate multiple conditions. For example, to find the sales of “Furniture” in the “West” region: =SUMIFS(Table1[Sales], Table1[Region], “West”, Table1[Category], “Furniture”).
    • Quote: “it will basically add a condition calculate the sum of sales where the region is equals to West simple to like or similar to seal query right so that is exactly what we’re going to do in Excel today that is using summs”

    10. Introduction to Microsoft Copilot in Excel:

    • Functionality: Microsoft Copilot is an AI-powered tool integrated into Excel designed to assist with data analysis and report creation.
    • Capabilities (based on the excerpt):Generating regional or country-wise sales reports based on text commands.
    • Identifying the day of the week an order was placed.
    • Potentially identifying customers with the highest sales (although an error occurred in the example).
    • Usage: Users can input natural language commands to Copilot to request specific analyses or reports.
    • Limitations (as observed): May experience issues with data type recognition (e.g., needing to explicitly format a column as “Text”). Can be slower compared to other AI modules. May encounter errors in certain data processing tasks (e.g., extracting text content from uploaded files).
    • Quote: “please help me create a regional wise sales report with this data set and can also give a few more commands right so it’s loaded right now you can see the icon here now you can write down give me country wise sales report and just fire the command”

    11. Power Query for Data Transformation:

    • Purpose: Power Query (Get & Transform Data) is a powerful tool in Excel for importing, cleaning, and transforming data from various sources.
    • Data Sources: Can import data from Excel files, XML, JSON, PDF, cloud services, SQL databases, and more.
    1. Workflow:Connect to Data Source: Select the data source (e.g., “From Folder”).
    2. Transform Data: Open the Power Query Editor to perform transformations such as:
    • Removing columns.
    • Changing data types.
    • Splitting columns by delimiters.
    • Renaming columns.
    • Performing calculations (e.g., calculating profit by subtracting cost from revenue).
    1. Load Data: Load the transformed data into an Excel worksheet or the Data Model.
    • Example Transformations: Changing the data type of “Order Date” and “Delivery Date” to Date format. Splitting a “Customer ID and Name” column into two separate columns using a space as a delimiter.
    • Quote: “Power Query window will shortly open now with the power query window opened you can select the orders 2022 Excel folder and it will be loaded now let’s try to cck click on the content button over here which should load our data”

    12. Understanding Time Series Data and Forecasting:

    • Definition: Time series data is a sequence of data points recorded at specific and fixed time intervals. It is time-dependent, and analysis often involves forecasting future values based on past trends.
    • Graphical Representation: Time series data is typically visualized as a line graph with time on the x-axis and the measured variable on the y-axis.
    • Quote: “time series data is basically a sequence of data that is recorded over a specific intervals of time”
    • Components of Time Series Data:
    • Trend: The overall long-term direction of the data (increasing, decreasing, or stable).
    • Seasonality: Periodic fluctuations in the data that occur at regular intervals (e.g., annually, monthly, weekly).
    • Cyclicity: Longer-term fluctuations that are not strictly periodic (e.g., business cycles, recessions). The duration between cycles is typically longer and less fixed than seasonality.
    • Irregularity (Random Component): Random, unpredictable variations in the data that cannot be attributed to trend, seasonality, or cyclicity.
    • Quote: “Time series data consists of primarily four components one is the trend then we have the seasonality then cyclicity and then last but not least regularity or the random component”
    • Stationary vs. Non-Stationary Data:
    • Stationary Data: Data whose statistical properties (mean, variance) remain constant over time. Many time series models assume stationarity.
    • Non-Stationary Data: Data exhibiting trends or seasonality, where statistical properties change over time. Transformations (e.g., differencing) are often needed to make non-stationary data stationary before modeling.
    • Quote: “if you have taken the raw data this is how it would look and uh what do you think it is is it stationary no right because there is a trend upward Trend so this is not a stationary data”
    • Moving Average Method: A simple forecasting technique that smooths out fluctuations in the data by calculating the average of a fixed number of preceding data points.
    • Centered Moving Average: A moving average where the average is associated with the midpoint of the time period, further smoothing the data.
    • Quote: “we will just go ahead and uh manually do the forecasting using what is known as moving average method”
    • Multiplicative Model: A time series model where the components (seasonality, trend, irregularity) are multiplied together to represent the data.
    • Formula (simplified for prediction): Predicted Value = Seasonality × Trend × Irregularity
    • ARIMA Model for Time Series Forecasting (Introduction):
    • Acronym: Autoregressive Integrated Moving Average.
    • Parameters: Specified by three parameters (p, d, q):
    • p (Autoregressive): Number of lagged values of the dependent variable used in the model. The current value is regressed on its own past values.
    • d (Integrated): Number of times the data needs to be differenced to become stationary.
    • q (Moving Average): Number of lagged forecast errors used in the model.
    • Assumption: ARIMA models typically assume that the time series data is stationary.
    • Quote: “we will be using the arima model to do the forecast of uh this time series data so let us try to understand what is arima model so arima is actually an acronym it stands for autor regressive integrated moving average”
    • Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF): Tools used to analyze the correlation between a time series and its lagged values, helping to determine the parameters (p and q) of an ARIMA model.
    • Autocorrelation: The correlation between values of the same variable at different time points.
    • Partial Autocorrelation: The correlation between a time series and its lagged values, after removing the effects of the intermediate lags.
    • Quote: “in order to test whether the data is stationary or not there are two important components that are considered one is the autocorrelation function and other is the partial autocorrelation function so this is referred to as ACF and pacf”
    • Forecasting in R using ARIMA (Brief Introduction): The excerpt briefly mentions using the forecast package in R to implement ARIMA models for forecasting. It highlights the auto.arima function, which automatically selects the optimal parameters (p, d, q) for the ARIMA model based on information criteria like AIC (Akaike Information Criterion). Model diagnostics and validation using tests like the Ljung-Box test are also mentioned.

    13. Creating Dashboards in Excel:

    • Definition: A visual interface that provides an overview of key performance indicators (KPIs) and relevant data using charts and graphs to facilitate quick understanding and decision-making.
    • Types: Strategic, analytical, and operational dashboards.
    • Advantages: Quick detection of outliers and correlations, comprehensive data visualization, and time-saving compared to running multiple reports.
    • Demo using Sample Sales Data (overview): The excerpt introduces the intention to create dashboards using a sample sales dataset, implying the use of charts and potentially PivotTables to visualize sales performance by various dimensions (e.g., region).
    • Quote: “a dashboard is a visual interface that provides an overview of key measures relevant to a particular objective with the help of charts and graphs”

    14. Descriptive Statistics using Excel’s Data Analysis Toolpak:

    • Accessing the Toolpak: Requires enabling the “Analysis Toolpak” add-in (File > Options > Add-ins > Excel Add-ins > Go…).
    • Descriptive Statistics Feature: Under the “Data” tab, the “Data Analysis” option appears after enabling the toolpak, providing access to various statistical analysis tools, including “Descriptive Statistics.”
    • Output: The “Descriptive Statistics” tool generates a table summarizing key statistical measures for selected data ranges, including:
    • Mean: Average of all data values.
    • Standard Error: Measure of the variability of sample means.
    • Median: Middle value in a sorted data set.
    • Mode: Most frequently occurring value.
    • Standard Deviation: Measure of the spread of data.
    • Sample Variance: Square of the standard deviation.
    • Kurtosis: Measure of the “tailedness” of the distribution.
    • Skewness: Measure of the asymmetry of the distribution.
    • Range: Difference between the maximum and minimum values.
    • Minimum: Smallest value.
    • Maximum: Largest value.
    • Sum: Total of all values.
    • Count: Number of data points.
    • Usage: Select the input range, specify the output options (e.g., new worksheet), and check the “Summary statistics” box to generate the descriptive statistics table.
    • Quote: “moving on the first value we got is the mean basically it is the average of all the data values […] moving on we have the second one which is standard error it is nothing but the measure of the variability of sample means”

    15. Using Chat GPT for Excel Tasks:

    • Capabilities Demonstrated:Identifying and Removing Duplicates: Can analyze an Excel spreadsheet and identify duplicate rows based on specified columns. Can provide options for handling duplicates (e.g., keeping the first or last occurrence).
    • Data Cleaning: Can find and remove extra spaces in cells. Can treat blank cells by filling them with placeholder values.
    • Data Analysis and Visualization: Can create PivotTables and corresponding charts (e.g., bar charts) to analyze data, such as sales performance by region. Can guide users through the process.
    • Advanced Analysis and Automation (Forecasting): Can assist in building forecasting models based on historical data, including steps like data preparation, model selection (e.g., ARIMA/SARIMA implied), and model evaluation. Can provide insights into time series characteristics (trend, seasonality) and perform statistical tests (e.g., ADF test for stationarity).
    • Interaction: Users interact with Chat GPT through text prompts, providing the Excel file and specifying the desired tasks.
    • Limitations Observed: May encounter errors during analysis or visualization (e.g., character encoding issues, plotting forecast confidence intervals).
    • Overall Value: Chat GPT can be a valuable tool for Excel users, assisting with various data-related tasks, enhancing productivity, and helping uncover insights.
    • Quote: “can you help me identify and remove duplicates duplicates from this Excel spreadsheet from the provided Excel spreadsheet […] please create a pivot table and a corresponding chart to analyze sales performance by region […] can you assist me in building a forecasting model to predict future sales based on historical data”

    16. Introduction to Pivot Tables and Charts:

    • Pivot Tables: Powerful tools for summarizing and analyzing large amounts of data. Allow users to rearrange and aggregate data based on different fields, enabling multi-dimensional analysis.
    • Pivot Charts: Visual representations of PivotTable data, providing interactive ways to explore trends and patterns.
    • Creating Basic Charts (Line Chart Example): The excerpt demonstrates creating a line chart to visualize month-on-month sales data. The steps involve selecting the date and sales data, going to the “Insert” tab, and choosing a line chart.
    • Creating Pivot Tables (Steps): Highlight the data range (including headers), go to the “Insert” tab, select “PivotTable,” confirm the data range and location for the PivotTable. Drag fields to the “Rows,” “Columns,” “Values,” and “Filters” areas to structure the analysis.
    • Grouping Dates in Pivot Tables: Excel can automatically group dates in a PivotTable by year, quarter, month, etc., facilitating time-based analysis.
    • Creating Pivot Charts from Pivot Tables: Select any cell within the PivotTable, go to the “PivotTable Analyze” tab (or “PivotTable Options” in older versions), and select “PivotChart” to choose a chart type.
    • Customizing Charts: Chart elements (legends, titles, axes labels) can be edited and customized for better presentation. Chart types can be changed after creation.
    • Quote: “please create a pivot table and a corresponding chart to analyze sales performance by region […] so what’s the first step highlight the range of cells that contain your data set and including headers and go back to Excel and you can just click on any cell and control a all your T set has been selected or these days you can just select one of the cells and go to insert menu and pivot table”

    17. Integration of Python in Excel:

    • Python Mode in Excel: Allows users to execute Python code directly within Excel worksheets.
    • Accessing Python: By typing =PY in a cell and pressing Tab, users can enter Python mode.
    • DataFrames: Python in Excel utilizes the Pandas library, with data from Excel ranges being represented as Pandas DataFrames.
    • Referencing Excel Ranges in Python: Excel ranges can be referenced within Python code (e.g., referencing a cell containing a data range).
    • Python Output in Excel: The results of Python computations can be displayed directly in Excel cells, either as Python objects (DataFrames) or converted to Excel values.
    • Basic Python Data Analysis in Excel (Examples):.describe(): Provides descriptive statistics for a DataFrame or a specific column.
    • .sum(): Calculates the sum of values in a column.
    • .mean(): Calculates the average of values in a column.
    • .groupby(): Groups data based on specified columns, enabling aggregated calculations (e.g., sum of sales per date or per month).
    • .plot(): Creates basic charts (e.g., line charts) within Excel.
    • Benefits: Leverages the power of Python’s data analysis libraries (Pandas) within the familiar Excel environment. Enables more complex data manipulation and analysis than standard Excel functions alone.
    • Quote: “you can see over this section that it is written the data range I mean the sales range and headers is equal to true as we have headers over here now press enter now you can see that it didn’t happen anything only it went to the next line for me to run this we need to press control plus enter Control Plus enter and see what we got we got a data frame over here”

    18. Formulas vs. Functions in Excel:

    • Formula: An expression that you write in a cell that performs calculations or other actions on the data in your worksheet. It always starts with an equals sign (=) and can contain cell references, constants, operators, and functions.
    • Function: A predefined formula that performs specific calculations in a particular order using specified values (arguments). Excel has a vast library of built-in functions (e.g., SUM, AVERAGE, COUNT, VLOOKUP).
    • Key Difference: Formulas are user-defined calculations, while functions are built-in, ready-to-use calculation tools. Formulas often use functions as part of their logic.
    • Quote: “a formula is an expression that calculates the value of a cell. Formulas can perform simple calculations such as addition and subtraction as well as more complex operations using functions. A function is a predefined formula that performs calculations using specific values in a particular order or structure”

    19. Order of Operations in Excel (PEMDAS/PEDMAS):

    • Acronyms: PEMDAS (Parentheses, Exponents, Multiplication and Division, Addition and Subtraction) or PEDMAS (Parentheses, Exponents, Division and Multiplication, Addition and Subtraction). The order for multiplication and division (and addition and subtraction) is from left to right.
    • Excel’s Evaluation Order: Excel evaluates formulas based on this order:
    1. Parentheses ()
    2. Exponents ^
    3. Multiplication * and Division / (from left to right)
    4. Addition + and Subtraction – (from left to right)
    • Importance: Understanding the order of operations is crucial for writing correct formulas that produce the intended results. Using parentheses can override the default order.
    • Example: =A1*10+5/2 will be evaluated as (A1*10) + (5/2). To force addition first, use =(A1+5)*10/2.
    • Quote: “the Order of Operations is first it will calculate what is there in the parenthesis it will perform this function and then it will see if there are any exponents in it and if it is there it will do that calculation the next and after that it will see if there is any multiplication or division […] and then it will see there is any addition […] and then the last one will be the substraction”

    20. Key Text Manipulation Functions:

    • CONCATENATE (or & operator): Combines text from multiple cells into one cell.
    • Syntax: =CONCATENATE(text1, [text2], …) or text1&text2&…
    • Example: =CONCATENATE(A1, ” “, B1) or =A1&” “&B1 would combine the text in cell A1, a space, and the text in cell B1.
    • Quote: “the concatenate function is used to join two or more text strings together into one string. The syntax is very simple equals concatenate open parenthesis and then you specify the text strings that you want to join separated by commas close the parenthesis and you will get it all merged in one particular cell”
    • Text to Columns: A feature used to split a single column of text into multiple columns based on a delimiter (e.g., space, comma, tab) or a fixed width.
    • Steps: Select the column to split, go to the “Data” tab, choose “Text to Columns,” select the delimiter type, specify the delimiter, preview the result, and choose the destination for the split data.
    • Use Case: Useful for separating first and last names from a full name column, or splitting addresses into separate components.
    • Quote: “you need to select that particular cell and then go to the data Tab and then choose text to columns there will be an option where in you need to select text to columns”

    21. VLOOKUP Function (Introduction):

    • Purpose: VLOOKUP (Vertical Lookup) is used to find and retrieve data from a specific column in a table based on a lookup value in another column (the first column of the table).
    • (Detailed explanation of VLOOKUP syntax and usage is expected in subsequent parts of the full course.)
    • Quote: “vlookup function is used for looking up a piece of information in a table and extracting some corresponding data or”

    22. Report Formats in Excel:

    • Available Formats: When creating reports (often using PivotTables), Excel offers different layout formats for displaying the data:
    • Compact Form: Displays all row labels in one column, using indentation to show hierarchy. Saves horizontal space.
    • Outline Form: Displays each level of row labels in its own column. Can repeat item labels.
    • Table Form: Similar to Outline Form but does not indent subordinate items. Each field is in its own column.
    • Choosing a Format: The best format depends on the data structure and the desired readability of the report.
    • Quote: “Excel is basically used for reporting mainly used for reporting to management you have to extract different kinds of reports and what are the report formats that are available in Excel this is one of the uh important questions again uh for a beginner level and there are three formats basically compact form outline form table form”

    23. Conditional Logic with IF and SUMIF Functions:

    • IF Function (Detailed): Performs a logical test and returns one value if the test is TRUE and another value if the test is FALSE.
    • Syntax: =IF(logical_test, value_if_true, value_if_false)
    • Logical Test: Any expression that can be evaluated to TRUE or FALSE (e.g., A1>10, B2=”Yes”).
    • Nesting IF Functions: Multiple IF functions can be nested within each other to handle more than two possible outcomes.
    • Example: =IF(A1>20, IF(B1>40000, “Valid”, “Invalid Salary”), “Invalid Age”) (as interpreted from the description).
    • Quote: “the F function in Excel it actually performs a logical test and you will give a condition it performs the test and returns a value if the test evaluates to true and another value which you again will specify if the test result is false”
    • SUMIF Function (Detailed): Sums the values in a range that meet a specified criterion.
    • Syntax: =SUMIF(range, criteria, [sum_range])
    • range: The range of cells to evaluate the criteria against.
    • criteria: The condition that determines which cells in the range will be summed.
    • sum_range: (Optional) The range of cells to actually sum. If omitted, the cells in the range that meet the criteria are summed.
    • Example: =SUMIF(G2:G6, “>75000”, G2:G6) would sum the salaries in the range G2:G6 that are greater than 75,000.
    • Quote: “the sum IF function okay again the sum function adds the cell value specified by a given condition or a criteria so you are giving a condition and you are actually doing another function which is performing the addition”

    24. COUNTIF Function (Example):

    • Usage Example (implied): Finding the number of days where the number of deaths in Italy was greater than 200, using a COUNTIF function on a column containing death counts.
    • Syntax: =COUNTIF(range, criteria)
    • Example (hypothetical): If column G contains the number of deaths in Italy, =COUNTIF(G2:G100, “>200”) would count the number of cells in that range where the value is greater than 200.
    • Quote: “we will try to find the function so we’ll use the count function and count function is again simple count if you are specifying the column that is the uh source that you’re looking for that is G2 the column G and it is two it starts from two row two” (Incomplete sentence, but implies the use of COUNTIF with a range and criteria)

    25. Data Validation:

    • Purpose: Used to control the type of data that can be entered into a cell, helping to prevent errors and ensure data accuracy.
    • Accessing Data Validation: Select the cells where you want to apply validation, go to the “Data” tab, and click on “Data Validation.”
    • Validation Criteria: Various criteria can be set, such as:
    • Whole number
    • Decimal
    • List
    • Date
    • Time
    • Text length
    • Custom formula
    • Error Alerts: Custom error messages can be displayed when invalid data is entered.
    • Input Messages: Helpful messages can be shown to guide users on the expected data.
    • Example: Setting data validation for a cell to only allow whole numbers between 1 and 10. If a user tries to enter text or a number outside this range, an error alert will appear.
    • Quote: “Data validation is one of the very important features in Excel which is used to restrict the type of data or the values that users enter into a cell. To apply it you need to select the cells where you want to apply the data validation go to the data tab and click on data validation”

    26. Applying Conditional Logic (IF and AND):

    • Combining IF and AND: The AND function can be used within the logical test of an IF function to check if multiple conditions are all TRUE.
    • Syntax: =IF(AND(condition1, condition2, …), value_if_true, value_if_false)
    • Example (Student Pass/Fail): To determine if a student passes based on marks greater than 60 AND attendance greater than 75%: =IF(AND(U5>60, V5>75), “Pass”, “Fail”).
    • Quote: “you have to use the IF function and check with the and condition to fill the results column […] we are checking the conditions here and the marks should be in row I mean the column U and it starts with u5 so let’s see u5 okay it will immediately go to column U and five row five okay and then what is the condition we are satisfying here it should be greater than 60 okay and the other condition is uh the column V that is the attendance column okay and what is the condition we are specifying V 5 should be greater than 75”

    27. Calculating Age Using YEARFRAC and DATEDIF:

    • YEARFRAC Function: Returns the fraction of the year represented by the number of whole days between a start date and an end date. Can be used to calculate age as a decimal.
    • Syntax: =YEARFRAC(start_date, end_date, [basis])
    • basis: (Optional) Specifies the day count basis to use (e.g., actual/actual, 30/360).
    • DATEDIF Function (revisited for age): As mentioned earlier, can calculate the difference in whole years.
    • Example (YEARFRAC for age): =YEARFRAC(birth_date, today())
    • Example (DATEDIF for age): =DATEDIF(birth_date, today(), “Y”)
    • Quote: “use the ear fraction or dated IF function to return the number of whole days between start date and the end date […] here is a small example and this is how we use the ear Frack function”

    28. Nested IF Statements (Detailed):

    • Purpose: To test multiple conditions and return different values based on which condition is met.
    • Logic: One IF function is placed inside another IF function’s value_if_false argument (or sometimes value_if_true).
    • Example (Excellent/Bad/Average): =IF(B2>80, “Excellent”, IF(B2<=60, “Bad”, “Average”)). This formula first checks if the value in B2 is greater than 80. If TRUE, it returns “Excellent.” If FALSE, it proceeds to the nested IF function, which checks if B2 is less than or equal to 60. If TRUE, it returns “Bad”; otherwise, it returns “Average.”
    • Quote: “the IF function can be nested I mean it can be Loop when we have multiple conditions to meet okay it can be nested the false value is replaced by another if function”

    29. Descriptive Statistics Using Data Analysis Toolpak (Recap and Steps):

    1. Steps:Enable the Analysis Toolpak (File > Options > Add-ins > Go > Analysis Toolpak > OK).
    2. Go to the “Data” tab and click “Data Analysis.”
    3. Select “Descriptive Statistics” and click “OK.”
    4. Specify the “Input Range” (the data you want to analyze).
    5. Specify the “Output Options” (e.g., a new worksheet).
    6. Check the box for “Summary statistics.”
    7. Click “OK.”
    • Output Interpretation: The generated table provides key statistical measures like mean, median, standard deviation, etc., helping to understand the central tendency and dispersion of the data.
    • Quote: “to analyze the data so we might come across a question and you will be given a data or a table and you need to find the descriptive statistics of the columns using data analysis tool […] how you do that is you have to add a pack okay okay you should be knowing that you need to add a pack which is called the analysis tool pack which you go to uh file options and then just click on the addin and select the analysis tool pack click uh Excel addin and just click go then it will add the option and then when you select the data you have to go to the click on the data analysis option”

    30. Calculated Fields in Pivot Tables:

    • Purpose: Allow users to create new fields in a PivotTable based on calculations involving existing fields. This enables analysis of data that isn’t directly present in the original source.
    • Accessing Calculated Fields: Select any cell in the PivotTable, go to the “PivotTable Analyze” tab (or “Options” in older versions), click on “Fields, Items, & Sets,” and choose “Calculated Field.”
    • Formula Creation: A dialog box opens where you can enter a name for the new field and create a formula using the existing PivotTable fields and standard Excel operators.
    • Example (Bonus Calculation): Creating a “Bonus” field based on “Sales” and “Unit Sold” with a condition (e.g., if Unit Sold > 1000, Bonus = Sales * 5%, else Bonus = Sales * 2%). The formula might involve an IF function within the calculated field definition.
    • Quote: “Allow you to perform calculations on the data in your pivot table. These calculations can involve other fields in the pivot table and help you derive new insights from your data. To add a calculated field you need to go to the pivot table analyze tab click on fields items and sets and select calculated field”

    31. Slicers for Pivot Table Filtering:

    • Purpose: Visual filters that provide an easy and interactive way to filter the data displayed in a PivotTable.
    • Inserting Slicers: Select any cell in the PivotTable, go to the “PivotTable Analyze” tab (or “Options” in older versions), and click “Insert Slicer.” In the dialog box, check the fields you want to create slicers for.
    • Usage: Clicking on items within a slicer filters the PivotTable to show only the data related to those selected items. Multiple items can be selected (often using Ctrl or Shift keys). Multiple slicers can be used simultaneously for multi-dimensional filtering.
    • Example: Adding slicers for “Month” and “Country” to a PivotTable. Clicking “Feb” in the “Month” slicer and “USA” in the “Country” slicer would filter the PivotTable to show data only for February in the USA.
    • Quote: “Slicers are used to further filter data in the pivot table suppose you already have some data and it’s for ease that you can do it by just adding a slicer you can select particular uh data in or a field and you can see the output for that particular field that you have chosen in the slicer […] go to the insert Tab and select slicer under filters”

    32. What-If Analysis Tools (Goal Seek, Scenario Manager, Data Table):

    • Purpose: Allow users to explore how changes to certain input values in a formula affect the output. Useful for sensitivity analysis and planning.
    • Accessing What-If Analysis: Located under the “Data” tab, in the “Forecast” group, click the “What-If Analysis” dropdown.
    • Goal Seek: Allows you to find the input value needed for a formula to reach a specific target output value. Useful for answering “What input value do I need to achieve this result?” questions.
    • Parameters: Set cell (the formula cell), To value (the target output), By changing cell (the input cell to adjust).
    • Example: Finding out what interest rate is needed to achieve a specific monthly payment on a loan.
    • Quote: “Goal seek is a feature in Excel that allows you to adjust the value of one input cell in a formula to find the value needed to achieve a desired result in another cell which is the output cell. It’s particularly useful when you know the outcome you want but are unsure what input value is required to get there”
    • Scenario Manager: Allows you to define and save different sets of input values (scenarios) and see their impact on the formulas in your worksheet. More advanced than Goal Seek as it can handle multiple changing variables simultaneously. Provides a summary report of the different scenarios and their results.
    • Usage: Define different scenarios with varying input values, and the Scenario Manager will show the resulting output for each scenario.
    • Example: Analyzing the impact of different sales growth rates and expense levels on the company’s profit.
    • Quote: “the scenario manager it is a bit more complicated compared to the other two but then it is uh more advanced than goal seek as it allows you the uh to adjust the multiple variables at the same time”
    • Data Table: Allows you to see how one or two variables in a formula change the results of that formula. Creates a table of outcomes based on different possible input values.
    • Types: One-variable data table (examines the effect of one input variable on one or more formulas) and two-variable data table (examines the effect of two input variables on one formula).
    • Example (One-Variable): Calculating the monthly payment for a loan at different interest rates.
    • Example (Two-Variable): Calculating the total revenue based on different price points and sales volumes.
    • Quote: “data table is another what if analysis tool that allows you to see how changing one or two variables in your formula will affect the results of that formula it’s a way to automate multiple what if questions and display the answers in a table”

    33. Functions vs. Subroutines in VBA:

    • Function (VBA): A block of VBA code that performs a specific task and returns a value. Can be called from other VBA code or directly used in worksheet formulas (if written as a user-defined function – UDF).
    • Called by: A variable or directly in a formula.
    • Purpose: To perform calculations and return a result.
    • Example: A VBA function to calculate the area of a rectangle given its length and width, returning the calculated area.
    • Quote: “a function always returns a value of the task it is performing […] functions are called by a variable […] functions are used directly in spreadsheet as formulas”
    • Subroutine (Sub) (VBA): A block of VBA code that performs a specific task but does not directly return a value. Subroutines perform actions, such as manipulating worksheet data, displaying messages, or controlling application behavior.
    • Called by: Name from other VBA code, by triggering events (e.g., clicking a button), or through the Macro dialog box.
    • Purpose: To automate tasks and perform actions.
    • Example: A VBA subroutine that formats a selected range of cells with a specific font and color.
    • Quote: “sub routine it does not return a value of the task it is performing […] sub routines can be recalled from anywhere in the program in multiple types […] Subs cannot be used directly in spreadsheets as formula”

    This briefing document summarizes the key concepts and functionalities of Excel covered in the provided excerpts. It highlights the importance of understanding basic functions, data manipulation techniques, data analysis tools, visualization options, and the potential of advanced features like macros, Power Query, and integration with AI tools and programming languages like Python. The distinction between formulas and functions, the order of operations, and the difference between VBA functions and subroutines are also crucial for effective Excel usage.

    Frequently Asked Questions about Excel Based on the Provided Sources

    1. What is the COUNT function in Excel used for? The COUNT function in Excel is specifically used to count the number of cells within a selected range that contain numerical values (numbers). It is helpful for quickly assessing the density of numbers in a dataset, particularly when the data includes a mix of different types like text and numbers. The function ignores blank cells and cells containing text or other non-numeric data types. For example, =COUNT(J2:J5) would count how many cells in the range J2 to J5 contain numbers.
    2. How does the COUNTA function differ from the COUNT function? While the COUNT function only counts cells containing numbers, the COUNTA function counts all non-empty cells in a specified range. This includes cells with numbers, text, dates, logical values, and even cells containing formulas that result in any of these types of data. Importantly, like the COUNT function, COUNTA does not count blank cells. For instance, =COUNTA(K2:K5) would count all cells in the range K2 to K5 that are not blank, regardless of their content (number, text, date, etc.).
    3. What is the purpose of the COUNTIF function in Excel? The COUNTIF function in Excel is used to count the number of cells within a given range that meet a specific criterion or condition. This function is valuable for analyzing data based on certain requirements. For example, you could use COUNTIF to count how many times a specific product appears in a list or how many sales figures are above a certain value. The syntax for COUNTIF involves specifying the range to be checked and the criteria to be met.
    4. How can you change the format of dates in Excel? Excel offers several ways to change the format in which dates are displayed. One common method is to select the cells containing the dates, right-click, and choose “Format Cells.” In the “Format Cells” dialog box, under the “Number” tab, you can select “Date” from the category list. This will present you with various built-in date formats that you can choose from. Additionally, you can customize date formats by selecting “Custom” and using format codes (like YYYY for year, MM for month, DD for day) to arrange the date components as desired. You can also change the regional settings (e.g., from India to United States) to automatically apply the date format conventions of that region.
    5. How can you calculate age from a date of birth in Excel? To calculate the age in years from a date of birth, you can use the DATEDIF function (or sometimes referred to as DATE DIFF). This function calculates the difference between two dates based on a specified interval (like years, months, or days). The basic syntax is =DATEDIF(start_date, end_date, unit). For calculating age in years, the start_date would be the date of birth, the end_date would typically be today’s date (which can be entered using the TODAY() function), and the unit would be “Y” for years. For example, if the date of birth is in cell A2, the formula might look like =DATEDIF(A2, TODAY(), “Y”).
    6. What is standard deviation and how is it calculated in Excel? Standard deviation is a statistical measure that quantifies the amount of dispersion or spread of a set of data values around their mean (average). A low standard deviation indicates that the data points tend to be close to the mean, while a high standard deviation indicates that the data points are more spread out. In Excel, you can calculate the standard deviation of a sample using the STDEV.S function or the standard deviation of an entire population using the STDEV.P function. Both functions take a range of numerical values as their argument. For example, =STDEV.S(A1:A10) would calculate the sample standard deviation of the values in cells A1 through A10. The calculation of standard deviation conceptually involves finding the variance (the average of the squared differences from the mean) and then taking the square root of the variance.
    7. What is the SUMIFS function in Excel and how is it used? The SUMIFS function in Excel is used to calculate the sum of values in a range that meet multiple specified criteria. It allows you to apply more than one condition to determine which cells should be included in the sum. The syntax for SUMIFS is =SUMIFS(sum_range, criteria_range1, criteria1, [criteria_range2, criteria2], …). The sum_range is the range containing the values to be summed. Each subsequent pair of criteria_range and criteria defines a condition. For example, to find the total sales in the “West” region, you might use a formula like =SUMIFS(Sales_Column, Region_Column, “West”), assuming you have columns named “Sales_Column” and “Region_Column”. You can add more criteria by specifying additional criteria_range and criteria pairs. It’s often beneficial to format your data as a table in Excel before using SUMIFS for easier column referencing.
    8. What is Microsoft Copilot for Excel and what kind of tasks can it help with? Microsoft Copilot for Excel is an AI-powered tool designed to assist users with various Excel-related tasks. It can help with data analysis by generating regional-wise sales reports or country-wise sales reports based on a given dataset. Copilot can also assist in data transformation, such as identifying the day of the week for order placement based on an order date column or finding the customer with the highest sales record. Furthermore, it can aid in building forecasting models to predict future sales based on historical data. While noted as being potentially slower than other AI tools, Copilot aims to streamline workflows and provide insights directly within the Excel environment by understanding user commands and data patterns.

    Microsoft Excel Basics and Fundamentals

    Based on the source “003-Excel_Full_Course_2025___Excel-03-24-2025.pdf”, let’s discuss Microsoft Excel basics:

    What is Microsoft Excel? Microsoft Excel is a software product developed by Microsoft that is more than just a spreadsheet; it’s a powerful tool for turning data into insights that drive smarter business decisions. It is designed for storing data in an organized way using rows and columns. Excel is also capable of manipulating data through mathematical operations and can be used to extract insights from data and represent them visually using graphs and charts. In 2025, Excel is considered an essential skill in every data-driven industry.

    Fundamentals of Microsoft Excel: When you start Microsoft Excel, you will first see the Microsoft Excel homepage. This page offers various suggestions based on the type of sheet you want to work with, including blank workbooks and templates for various purposes like business, personal use, planners, trackers, lists, budgets, and charts.

    Once you open a sheet, you will encounter several key interface elements:

    • Toolbar Menu: Located at the top, this menu includes options like File, Home, Insert, Draw, Page Layout, Formulas, Data, Review, View, and Help. These tools are used to work on your data.
    • Toolbar Ribbon: When you select an option from the toolbar menu, a ribbon appears below it. This ribbon contains various options and functionalities specific to the selected tool. For example, selecting the “Home” tool displays a ribbon with options for clipboard functions (Paste, Cut, Copy, Format Painter), font manipulation (font type, size, bold, italic, underline), and text alignment. Each tool in the toolbar menu (Insert, Draw, etc.) has its own unique ribbon.
    • Toolbar Groups: The toolbar ribbon is further segmented into groups, where each group contains a set of related functions. For instance, the “Home” ribbon has groups like “Clipboard,” “Font,” and “Alignment,” each with specific operations. Some toolbar groups have a small arrow icon in the corner, which, when clicked, opens a dialogue box with more options that couldn’t fit in the main group section.
    • Cell and Address: An Excel sheet is made up of boxes called cells. Each cell has its own unique address, which is a combination of its column letter and row number. For example, a highlighted cell might have the address “B3,” where “B” is the column name and “3” is the row number. The cell address is displayed in a designated area of the Excel interface.
    • Sheet Tracker: Located in the bottom left corner of the Excel sheet, the sheet tracker allows you to navigate through different sheets within the same Excel file (workbook). You can add new sheets by clicking on a “+” option.
    • Sheet Size Control: In the bottom right corner, there is an option to increase or decrease the sheet size (zoom level).

    Basic Excel Functions and Formulas: Learning basic Excel functions and formulas is crucial for mastering Excel and can significantly boost productivity. These are fundamental for daily tasks and data analysis. Some essential basic functions include:

    • SUM: Used to add up numbers. It can calculate the total of values typed directly into the function or values within a range of cells. The formula starts with =SUM() followed by the range of cells (e.g., =SUM(A2:A5)).
    • IF: Performs a logical test and returns one value if the condition is true and another value if the condition is false. The formula structure is =IF(logical_test, value_if_true, value_if_false) (e.g., =IF(B2>300, “Above 300”, “300 or Below”)).
    • AVERAGE: Calculates the mean or average of numbers provided. The formula is =AVERAGE() followed by the range of cells (e.g., =AVERAGE(C2:C5)).
    • MIN and MAX: Used to find the minimum and maximum values within a given range of data, respectively. The formulas are =MIN(range) (e.g., =MIN(D2:D5)) and =MAX(range) (e.g., =MAX(E2:E5)).
    • TRIM: Removes extra spaces from text, except for single spaces between words. It’s useful for cleaning data from inconsistent sources. The formula is =TRIM(text) (e.g., =TRIM(F2)).
    • CONCATENATE: Joins two or more text strings into one string. It can combine data from different cells. The formula is =CONCATENATE(text1, text2, …) (e.g., =CONCATENATE(H2,” “,I2)) or you can use the ampersand operator (&) for the same purpose (e.g., =H2&” “&I2).
    • COUNT: Counts the number of cells within a range that contain numbers. There are related functions like COUNTA which counts cells with any content and COUNTBLANK which counts empty cells.

    Mastering these basic elements and functions is the foundation for using Excel effectively for data analysis, report creation, task automation, and building interactive dashboards.

    Essential Excel Functions Explained

    Based on the source “003-Excel_Full_Course_2025___Excel-03-24-2025.pdf” and our previous discussion, let’s delve deeper into the usage of several key Excel functions. The source emphasizes that functions and formulas are crucial for mastering Excel and are fundamental for daily Excel tasks. Learning even a few essential functions can significantly enhance your abilities.

    Here’s a breakdown of the usage of some basic but important Excel functions:

    • SUM Function:
    • The SUM function in Excel is used to add up numbers.
    • It provides a quick way to calculate the total of several values, whether typed directly into the function or included within a range of cells.
    • Example: To sum the values in cells A2 through A5, you would select the cell where you want the result to appear and enter the formula =SUM(A2:A5) in the formula bar.
    • IF Function:
    • The IF function in Excel is used to perform logical tests and return values based on the results of these tests.
    • It checks whether a condition is TRUE or FALSE and then returns one value for a TRUE result and another for a FALSE result. This is extremely useful for decision-making processes within your data set.
    • Example: To check if the value in cell B2 is greater than 300, and display “Above 300” if true and “300 or Below” if false, you would enter the formula =IF(B2>300, “Above 300”, “300 or Below”). This logic can also be applied to other cells by dragging the fill handle.
    • AVERAGE Function:
    • The AVERAGE function in Excel calculates the mean or average of numbers provided to it.
    • This function is particularly useful for quickly finding the central values in a set of data, which can be helpful in various statistical analyses.
    • Example: To find the average of values in cells C2 through C5, you would enter the formula =AVERAGE(C2:C5) in the desired cell.
    • MIN and MAX Functions:
    • The MIN and MAX functions in Excel are used to find the minimum and maximum values within a given range of data, respectively.
    • These functions are useful for quickly identifying the smallest and largest numbers in a data set, helping you to analyze different ranges efficiently.
    • Examples: To find the minimum value in cells D2 through D5, use =MIN(D2:D5). To find the maximum value in cells E2 through E5, use =MAX(E2:E5).
    • TRIM Function:
    • The TRIM function in Excel is used to remove all the extra spaces from text, except for a single space between words.
    • This function is especially useful for cleaning up data that comes from other sources or has been entered inconsistently.
    • Example: If cell F2 contains text with extra spaces, you can clean it by entering the formula =TRIM(F2) in another cell.
    • CONCATENATE Function:
    • The CONCATENATE function in Excel is used to join two or more text strings into one string.
    • It helps in combining data from different cells, which can be useful for creating full names, combining addresses, or any situation where text needs to be merged.
    • Example: If “Andrew” is in cell H2 and “Garfield” is in cell I2, you can combine them with a space by using the formula =CONCATENATE(H2,” “,I2) or =H2&” “&I2.
    • COUNT Function:
    • The COUNT function in Excel is used to specifically count the number of cells that contain numbers in a Range.
    • It’s a crucial function for quickly assessing the numbers density in a data set, especially when dealing with mixed data types.
    • Example: To count the number of cells containing numbers in the range J2 to J5, you would use the formula =COUNT(J2:J5).
    • COUNTA Function:
    • The COUNTA function in Excel counts the number of cells in a Range that are not empty.
    • It is useful for determining the size of data sets that include various types of data such as numbers, text, or even logical values.
    • Difference from COUNT: While COUNT only counts numeric values, COUNTA counts any non-empty cell. It does not count blank cells.
    • Example: To count all non-empty cells in the range K2 to K5, use =COUNTA(K2:K5). This will count cells containing numbers or text.
    • COUNTIF Function:
    • The COUNTIF function is used to count the number of cells within a range that meet a specified criteria.
    • This function is valuable for filtering and analyzing data based on specific conditions.
    • Example: To count the number of cells in the range L2 to L5 that contain a value greater than 5, you would use the formula =COUNTIF(L2:L5, “>5”).
    • SUMIF Function:
    • The SUMIF function in Excel allows you to add up values in a Range based on specified criteria.
    • This function is extremely useful for performing conditional sums where you want to sum numbers that meet certain conditions.
    • Example: To calculate the sum of values in the range M2 to M5 that are greater than 5, you would use the formula =SUMIF(M2:M5, “>5”).

    The source further highlights that Microsoft consistently updates Excel, introducing new functions to enhance data management capabilities. Mastering these basic functions lays the groundwork for understanding and utilizing more advanced features in Excel.

    Excel for Data Analysis: 2025 Essentials

    Based on the source “003-Excel_Full_Course_2025___Excel-03-24-2025.pdf” and our previous discussions, Microsoft Excel is presented as more than just a spreadsheet; it is a powerful and essential tool for data analysis in 2025. It enables users to transform raw data into valuable insights that can drive smarter business decisions. The source highlights that proficiency in Excel, from basic to advanced techniques, is vital for tasks such as creating reports, automating tasks, and building interactive dashboards. Furthermore, it mentions the exploration of advanced features like Power Query, Co-pilot for AI insights, and the use of key formulas to boost productivity.

    Here are key aspects of data analysis in Excel as discussed in the source:

    • Foundational Tool: Excel serves as a foundational tool for initial data exploration and basic analysis. Its versatility in data manipulation, visualization, and modeling is unmatched at this stage.
    • Data Organization and Manipulation: Excel’s fundamental structure of rows and columns allows for organized data storage. It is also capable of manipulating data through various mathematical operations.
    • Essential Functions for Analysis: The source emphasizes the crucial role of functions and formulas in mastering Excel for data analysis. Basic but important functions like SUM, AVERAGE, IF, COUNT, and SUMIF are fundamental for daily Excel tasks and make handling data easier and more efficient. Our previous discussion elaborated on these and other basic functions like MIN, MAX, TRIM, CONCATENATE, and COUNTIF, which are all building blocks for analyzing data.
    • Data Visualization: Excel allows users to represent data in visually appealing graphs and charts, which is crucial for extracting insights and communicating findings effectively. The source demonstrates the creation of various chart types like pie charts, column charts, bar charts, and line graphs for different analytical purposes.
    • Interactive Dashboards: A significant aspect of data analysis in Excel is the ability to build interactive dashboards. These dashboards allow for dynamic exploration and presentation of data using elements like pivot tables and slicers.
    • Data Filtering and Sorting: Excel provides robust features for filtering data to focus on specific subsets and sorting data to identify trends and patterns, including sorting by date and multiple criteria. Advanced filtering options are also available for more complex data extraction.
    • What-If Analysis: Excel offers what-if analysis tools like Goal Seek to experiment with different scenarios and understand how changes in input variables can affect outcomes.
    • Data Cleaning: Preparing data for analysis is crucial, and Excel provides tools for removing duplicate values and using functions like TRIM to clean text data.
    • Advanced Analysis Features: The source introduces more advanced capabilities for data analysis:
    • Power Query: For importing and transforming data from various sources.
    • Power Pivot: An Excel add-in for analyzing large data sets from multiple sources and creating relationships between tables.
    • Co-pilot for AI Insights: Integrating AI to assist with data analysis tasks.
    • Data Analysis Toolpak: An add-in that provides advanced data analysis tools for statistical calculations, including descriptive statistics like mean, median, mode, standard deviation, etc..
    • Python in Excel: Integration of Python for advanced data analysis, visualization, and automation within the Excel environment.
    • Pivot Tables: Pivot tables are highlighted as essential tools for summarizing and analyzing large datasets. They allow for easy aggregation, filtering, and comparison of data, and can be used to create pivot charts for visual representation. The source demonstrates creating pivot tables with multiple variables and calculating percentage contributions.
    • Integration with Other Tools: While Excel is powerful, the source also acknowledges that data analysts often use it in conjunction with other tools like SQL, Python, Tableau, and Power BI for more advanced tasks.

    In summary, the source “003-Excel_Full_Course_2025___Excel-03-24-2025.pdf” positions Microsoft Excel as a comprehensive tool for data analysis, offering a wide range of features from basic data organization and manipulation to advanced statistical analysis, visualization, and integration with other powerful platforms and languages. It emphasizes the continuous evolution of Excel with the introduction of features like Co-pilot and Python integration, further solidifying its role in the modern data-driven landscape.

    Data Visualization Tools: Excel, Tableau, Power BI, and Python

    Based on the source “003-Excel_Full_Course_2025___Excel-03-24-2025.pdf” and our conversation history, several tools for data visualization are discussed:

    • Microsoft Excel: The source extensively highlights Excel’s capabilities as a powerful tool for turning data into insights, with a strong emphasis on data visualization through various types of charts and graphs. Excel allows users to represent data in visually appealing formats, which is crucial for understanding trends, patterns, and relationships within the data. The document demonstrates the creation of pie charts, column charts, bar charts, and line graphs to represent different types of data, such as company shares, profits over time, and revenue growth. Interactive dashboards can be built in Excel using pivot tables and pivot charts combined with slicers to filter data dynamically, making complex data more understandable. The source emphasizes that data visualization skills are paramount for data analysts as data complexity grows, and the ability to present insights clearly and persuasively is essential.
    • Tableau and Power BI: The source mentions Tableau and Power BI as examples of data visualization tools that data analysts should be proficient in. These are presented alongside Excel, MySQL, and programming languages like Python and R, suggesting they are important components of a data analyst’s toolkit. The source implies that these tools are used for more advanced data visualization, although specific details about their functionalities are not provided within the excerpts.
    • Python: The source identifies Python programming language as essential for data analysts, enabling data manipulation, advanced statistical analysis, and machine learning implementations. While the primary focus mentioned is not solely visualization, it’s widely known that Python has powerful libraries like Matplotlib and Seaborn that are extensively used for creating various types of static, interactive, and animated visualizations in data analysis [information not in the source, please verify independently]. The source briefly touches upon using Python within Excel, showcasing the creation of a line chart as a Python output.

    In summary, the source emphasizes Microsoft Excel as a key data visualization tool, equipped with comprehensive charting features and the ability to create interactive dashboards. Tableau and Power BI are mentioned as other important data visualization tools for data analysts. Additionally, Python is highlighted for its broader data analysis capabilities, which include powerful visualization libraries [information not in the source, please verify independently], and its integration within Excel is also noted. The ability to present insights clearly and persuasively through data visualization is considered a core skill for data analysts in the context of data-driven communication.

    Becoming a Data Analyst: Skills, Qualifications, and Experience

    Based on the source “003-Excel_Full_Course_2025___Excel-03-24-2025.pdf” and our conversation history, becoming a data analyst involves acquiring a specific set of skills, qualifications, and practical experience. The source outlines a five-step approach to becoming a data analyst: focusing on skills, obtaining proper qualification, testing skills through personal projects, building a portfolio, and targeting entry-level jobs or internships.

    Here’s a breakdown of these aspects based on the source:

    1. Essential Skills:

    The source categorizes the necessary skills into six key areas:

    • Data Cleaning: Ensuring data is accurate and ready for analysis.
    • Data Analysis: Extracting meaningful insights from data. Proficiency in Excel remains vital for data manipulation, visualization, and modeling as a foundational tool for initial data exploration and basic analysis. Excel is capable of manipulating data through mathematical operations and extracting insights. Basic Excel functions like SUM, AVERAGE, IF, COUNT, and SUMIF are essential.
    • Data Visualization: Presenting data insights clearly and persuasively. Excel offers tools to represent data in visually appealing graphs and charts. The source also mentions data visualization tools like Tableau and Power BI as important for data analysts.
    • Problem Solving: Identifying issues, formulating hypotheses, and devising innovative solutions for complex data challenges.
    • Soft Skills: Effective communication of findings to both technical and non-technical stakeholders, teamwork, and collaboration within multidisciplinary teams.
    • Domain Knowledge: Understanding the specific industry or domain in which one is working to provide accurate results.

    2. Technical Skills:

    Beyond the core skills, specific technical proficiencies are crucial:

    • Microsoft Excel: As highlighted, proficiency in Excel is vital for data manipulation, visualization, and modeling. The source details various Excel functionalities like formulas (e.g., COUNTIF, SUMIF), charting, creating interactive dashboards using pivot tables and slicers, Power Pivot for large datasets, Power Query for data transformation, Co-pilot for AI insights, and even the integration of Python for advanced analysis.
    • Database Management: Skills in database management, particularly proficiency in DDB systems and querying languages like SQL, are indispensable for accessing and manipulating data seamlessly.
    • Statistical Analysis: Understanding statistical concepts and methods to uncover trends, patterns, and correlations within data, facilitating evidence-based decision-making. The source mentions tools like the Data Analysis Toolpak in Excel for statistical calculations, including ANOVA and descriptive statistics.
    • Programming Languages: Proficiency in programming languages like Python is essential for advanced data manipulation, statistical analysis, and machine learning implementations. The source also mentions R programming language as another tool in a data analyst’s arsenal.
    • Data Visualization Tools: Familiarity with dedicated data visualization tools like Tableau and Power BI is important for creating more sophisticated and interactive visualizations.

    3. Qualifications:

    Formal education and training can significantly enhance one’s prospects of becoming a data analyst:

    • Master’s Courses, Online Courses, and Boot Camps: These provide structured learning to gain in-depth knowledge and specialized skills in data analysis.
    • Master’s Programs: Offer comprehensive, academically rigorous training, often including research projects.
    • Online Courses: Provide flexibility to learn at one’s own pace while covering essential topics.
    • Boot Camps: Offer immersive, hands-on training in a short period, focusing on practical skills.

    These qualifications enhance credibility, keep individuals updated on industry trends, and make them more attractive to potential employers.

    4. Practical Experience:

    Demonstrating practical skills is crucial for landing a data analyst role:

    • Data Analyst Projects: Working on personal projects demonstrates practical skills in data cleaning, visualization, and analysis. These projects help build a portfolio showcasing expertise and problem-solving abilities and bridge the gap between theory and real-world application.
    • Portfolio: A portfolio serves as tangible proof of data analysis skills and expertise through real-world projects, showcasing the ability to analyze and interpret data effectively. It highlights domain knowledge and problem-solving skills, providing a competitive edge.
    • Internships: Internships offer hands-on experience with real-world datasets, tools, and workflows, bridging the gap between theoretical knowledge and practical application. They provide exposure to industry practices, build professional networks, enhance resumes, and improve chances of securing full-time roles.

    5. Career Progression:

    The initial steps often involve gaining practical experience and exposure:

    • Targeting Entry-Level Jobs or Internships: These provide exposure to real-world data problems and allow for the application of learned skills in a professional setting.

    In conclusion, becoming a data analyst in 2025, as outlined by the source, requires a combination of strong technical skills in areas like Excel, SQL, and potentially programming languages and visualization tools, alongside essential soft skills and domain knowledge. Obtaining relevant qualifications through various educational paths and gaining practical experience through personal projects, a strong portfolio, and internships are critical steps in this career journey. The source emphasizes that proficiency in Excel remains a foundational and vital skill for aspiring data analysts.

    Excel Full Course 2025 | Excel Tutorial For Beginners | Microsoft Excel Course | Simplilearn

    The Original Text

    hello everyone and welcome to excel full course by simply learn Excel is more than just a spreadsheet it’s a powerful tool for turning data into insights that drivve smarter business decisions in 2025 Excel is an essential skill as data leads every industry in this course you learn from Basics to Advanced Techniques learning how to create reports automate task and build interactive dashboards you’ll also explore power query co-pilot for AI insights and 10 key formulas to boost productivity plus we’ll dive into dashboard creation charb automation database management and many more so let’s get started before we comment if you are interested to supercharge your career in data analytics the professional certificate program in data analytics and generative AI by E and ICT Academy it goari is your perfect choice this 11 Monon live online course offers interactive m master classes by I goti faculty and Industry experts combining Cutting Edge tools like generative AI chbt python Tab and SQL plus you’ll also earn an executive aluminize status from it goti and ibam recognized certifications to stand out in the job market Microsoft Excel so Microsoft Excel is a software product designed and developed by Microsoft storing data in an organized way that is rows and columns and Microsoft Excel is also capable to manipulate data through some mathematical operations followed by that Microsoft Excel is also used to extract the insights from the data and represent it in the form of visually appealing graphs and charts now we have a basic understanding or an overview of what a Microsoft Excel software product is now moving ahead we will understand the fundamentals of Microsoft Excel so following are some fundamentals that you need to know before getting started with Microsoft Excel so basically when you install the Microsoft Office in your computer you will have various Microsoft products out of which Microsoft Excel is one of the product so we will be dealing exactly with that particular product that is Microsoft Excel so when you get started with Microsoft Excel this is what you will see in the first page so this particular page is called as Microsoft Excel homepage where you will be having various varieties of sheets Microsoft Excel will give you some suggestions based on the type of sheet you want to work with we will see this in a better way through the Practical session so once you get started with the sheet you will have some more options so this particular option is called as the toolbar menu you will have the file home insert draw past layout formulas data review View and help so these are the tools that you will be using to work on your data using Microsoft Excel furthermore we have a toolbar ribbon so when you select some or the other option from the file home insert draw page layout formulas data review View and help buttons you will have a ribbon so for example you can see that I have selected the Home Tool here so when I press on the Home Tool this is the ribbon which Microsoft Excel gives me so this ribbon has some options in it which can perform various operations now in a further more detailed way we will have toolbar groups so when you see in the previous slide we have a complete toolbar ribbon so this particular ribbon been is segmented into groups so you can see the first group as paste Cut Copy format painter etc etc and the second group is the font the size of the font and to increase the size of the font to decrease the size of the font bold italic underline etc etc and here you can see the text alignment so each and every group has separate functions so each set is called as a group and I think you can see a small Arrow op over here so this Arrow option is used into the toolbar groups when the group is not able to fit all the operations or all the functionalities in one single provided section so when you click on this particular Arrow Mark you will have another dialogue box so this is called toolbar more options so you can see that when I clicked on this icon you can see a new dialogue box which opens me a new set of operations which are not able to be fit in this particular group so we will also see more about this in a better way in the practice iCal session now moving forward we have cell and address so when you open a Microsoft Excel sheet you can find boxes so each and every box is named as a cell and each cell has its own address for example the highlighted cell over here has an address B3 so B is the column name and three is the rule name and apart from that you can have the sheet tracker in the bottom left corner of the Excel sheet where you can navigate through different sheets and in the bottom right corner you have an option of increasing or decreasing the sheet size so these are the basic fundamentals of Microsoft Excel that you need to keep in mind before getting started so we will have more on this in the Practical session if you are new to Excel or looking to improve your skills learning these functions can really help Excel isn’t just making spreadsheets it’s a powerful tool for analyzing data and managing projects in this guide we will cover some basic but important Excel functions like sum average if count and sum if you’ll see how these functions do and learn and how to use them with practical examples this knowledge will help you with daily task and make you more confident in using Excel so let’s explore these functions together and see how they can make handling data easier and more efficient let’s get going and make the most of your data with Excel why learn basic Excel functions and formulas functions and formulas are crucial for mastering Excel by learning even a few essential ones you can outpace many around you they are fundamental for daily Excel task to deepen your knowledge or explore Advanced functions consider exploring a comprehensive database of all existing Excel functions Microsoft consistently updates Excel introducing new functions like xlup in Microsoft 365 enhancing your data management capabilities so now let’s move on to the demo part okay so first we will understand the sum function in Excel as we know the sum function in Excel is used to add up the numbers it provides a quick way to calculate the total of several values whether they are typed into a function directly included within a range of cells so these are the cells so you just have to type the numbers in the cell and uh give the function over here and it will give you the desired result let’s check how so first we will understand the sum function I’m just I’ll just write it over here so the first step is like open your Excel file in your computer and select any column or row like here I’m selecting the column for instance you might just put them in cells A1 till A5 so uh in the A1 part I’ve mentioned some for your understanding so we’ll start with A2 we’ll type all the values over here so just type any values like I’m typing 40 then we have 78 then 35 and then we have 67 from A2 till A5 we have typed the values and using the formula in the formula bar it will add all the numbers from cell A2 till A5 and it will display in this particular cell so just select the cell where you want your answer to be shown and just insert the formula over here the formula for sum function is equal to sum A1 till A5 so we just go back to this bar and just select this particular cell and here in this bar just type the formula A1 till A5 but then here we have A2 so we’re just going to change it to A2 till A5 all right and just press enter so here you get the value 220 which is the sum of all the values which we have mentioned over here so it was pretty simple right now let us understand the IF function in Excel okay so the IF function in Excel is used to perform logical test and return values based on the results of these test this function checks whether a condition is true or false and then returns one value for a True Result and another for a false result it is extremely useful for decision making process within your data set let us now understand what are the steps to use this if function so just input your data and put any numeric values in the cells from B2 till B5 okay so 56 90 36 23 and here I want my result to be shown so I’ll just insert the formula here in the function bar so okay just check out the formula with IF function which is equal to if A1 is greater than a00 above 300 300 or below so let’s see how it works we’re just going to enter the formula okay so this will tell Excel to check if the value of B2 is greater than 300 if true it will display above 300 and if false it will display 300 or below let’s check it out okay so there’s some mistake going to paste the formula here again so in place of A1 we will mention B2 we just type B2 over here if GR greater than 300 above 300 or 300 below just press enter so we’ll just enter the formula over here and in place of A1 we will provide B2 all right so we just type here B2 that’s it and we just press enter so you can see the result has come to 300 or below because the value is less than 300 and you can also so drag and fill handle from B1 down to B5 and apply the same logic in other columns as well and it will give you the same result let us now understand the next function which is the average function in Excel the average function and okay so what is this average function used for basically the average function in Excel it will calculate the mean or average of numbers provided to it this function is particularly useful for quickly finding the central values in a set of data which can be helpful in various statistical analysis let us now understand the steps to use the average function so the first step is like you just click on the cell wherever you want your value to appear all right so just provide any values suppose um let’s provide some bigger values like bigger numbers like 430 here we type 2 78 then we have 98 and we have uh 79 and here I want my value to be shown so I’ll just insert the average formula which is let’s check out the formula which is equal to equal to average what is the um column we have chosen from C2 so we’ll just type C2 over here till C2 till C5 C5 all right and we will just press enter so as you can see the average function value is shown over here so after pressing enter the Excel will compute and display the average of all the cells in the selected Cel the next function is the minimum and maximum functions in Excel it’s very very simple let’s discuss what is the minimum and maximum function used for the Min and the max functions in Excel are used to find the minimum and maximum values within a given range of data these functions are useful for quickly identifying the smallest and largest numbers in a data set helping you to analyze different ranges efficiently so let us now understand what are the steps to use this formula it’s very simple just type any values over here 500 next we have 389 then we have 200 190 198 and here I want my value to be shown the minimum value so in the function bar I’ll just press I’ll just type the function which is equal to Min and in the bracket you just mention the column which is starting from D2 right so D2 D2 till D5 so just type the formula over here and after pressing enter you can see the minimum value shown over here is 190 and it’s similar for the maximum function as well so for the max let’s let’s uh use some different values and I want my uh function to be shown here so same you have to use the same formula just type the in place of Max we’re using in place of minimum we using the max function so just type Max and then the column name which is from uh I think G no no no yeah e so just so in place of we will just write right e okay sorry we just type E2 till I think it’s still E5 right E5 that’s it and just press enter so you can see the maximum value shown over here that is 800 so it’s very simple the Exel will display the largest and the minimum value using Min and Max functions the next is to understand the trim function in Excel let us now understand the trim function in Excel so basically this trim function in Excel is used to remove all the extra spaces from text except for a single space between words this function is specially used for cleaning up the data that comes from other sources of data that has been entered inconsistently let us now understand the steps to use the STM functions it is same as we have seen for all these functions the only difference is that this stream function will exclude the spaces so assume your text Data with some Extra Spaces in cells suppose over here I type any sentence any text okay like hello and then I provide some extra space every one and suppose I want my value to be inserted over here to remove all the extra space or if you want some bigger text you can do that but let us now check how the trim functions work for this text just click on the cell where you want your clean text to appear that is F6 and using the function over here just write uh equal to trim and then in place of you just have to mention over here which is uh F2 which is F2 just press enter now you can see that I’ve provided some Extra Spaces over here but here using the trim function it will remove all the extra space and just make it in a clean manner so you can also drag the fill handle from B1 like from this F2 down to G2 and all of that and then you can use a trim function in other column as well to clean up the text the second function is the concatenate function so let’s discuss the concatenate function over here okay so this concatenate function in Excel is used to join two or more text strings into one string it helps in combining data from different cells which can be useful for creating full names from first and last names combining addresses or any situation where text needs to be merged so let us now look the step steps how to use this concatenate function so just assume that you have written your name suppose I’ve written Andrew in this column H2 and here I have written car field now if I want what if I want to combine Andrew Garfield together so just by using this concatenate function I can make it happen all right so if I want my uh result to be shown in this particular cell I’ll just click on this and then in the function bar I’ll just enter the formula which is which is uh equal to cona concatenate and then I have to mention the cell name which is H this is H2 right H2 H and this one is I2 so I’ll just mention the yeah and here and here after mentioning okay I have to provide some space over here so I’ll just put comma and then some space then comma and then here I’ll just I2 all right and here I’ll just press enter so you can see in this particular Rule and both name has been combined concatenated together into a single name so this is how we use the concatenate function and the next function we have is the count function in Excel let us now understand how to use this count function okay the count function in Excel is used to specifically count the number of cells that contain numbers in a Range it’s a crucial function for quickly assessing the numbers density in a data set especially when dealing with mixed data types we’ll just assume that you have added transaction counts in column J starting from column G2 till J5 so the transaction counts are as follows 7 89 0 100 till J5 and then click on the cell where you want your count to appear say J j6 and then you just go to the formula bar and just press just enter the formula over there which is equal to count bracket this should be J2 J2 in J5 and just press enter and you can see the Excel will display the number of cells that contain numbers in column J which is one 2 3 4 okay so this was all for count concatenate trim Max Min average if and Su functions next we have is the count a function let us now understand what is this count a function fun used for and how is it different from count function so this count a function in Excel counts the number of cells in a Range that are not empty it is useful for determining the size of data sets that include various type of data such as numbers text or even logical values so just it will just count the number of nonempty cells in a specific column just click on the cell where you want the count to appear okay let me just tell you the difference between count and count function is that the count function generally used to count a range of cells containing numbers or dates excluding blank whereas this count a function will count everything including numbers dates text or a range containing a mixture of these items but does not count blank cells so suppose if I enter 56 and I leave it blank and then I enter 98 and then again I leave it okay and then just enter 56 and I want my formula to be shown over here so I’ll just go in this formula bar and just enter equal to count a and then from K column this is K column right K2 K2 till K5 okay and I just press enter so as you can see it’s showing three one 2 3 it will not count the extra space or the empty call empty cell we have left behind and also one more thing that we can also type text over here it’s not necessary to type only the numbers you can also type the text for example I type Apple your right type banana now okay here is the function just press enter will count three which is 1 2 and three so this count a function is used both for text as well as the dates as well as the numeric values but the count function is only used for the numeric values and it it will not count the text over here now let us understand the count if function in Excel we have now understood count and count a function and what is the difference between both both of these function we will now understand the countif function over here so let’s understand the count if function okay so this counter function is used to count the number of cells in a Range that meet a specified criteria this function is valuable for filtering and analyzing data based on specific conditions to count the number of transaction that ex exceeds a certain amount of value a threshold value for example five just click on the cell and suppose you want your values to appear in column L2 so suppose I type here 89 7 3 and five and here I will enter the formula which is equal to count if and then from L2 till L5 okay just close the bracket and here you just also have to input the threshold value suppose I take the threshold value as uh let’s say three or let’s say five okay let’s take it five so this is my threshold value greater than five so it will display only the numbers which are greater than five and I just press enter so you can see it’s showing two okay because 7 and 89 are the only two values which are greater than five so you can see that using any threshold function even if I could have written over here let’s say three so it will show three because the number is greater than three are 5 7 and 89 so this is how count a function works we will now understand the last function which is the sum IF function in Excel so what is the sum of function used for basically the sum of function in Excel allows you to add up values in a Range based on specified criteria this function is extremely use useful for performing conditional sums where you can want to show the sum numbers that meet certain condition so to calculate the sum of transactions where the number is greater than five click on the cell where you want your total number to appear let’s say over here I select which is m6 and you just have to enter the formula before that just write any number values like 7 5 3 and 10 and over here just mention the formula which is equal to sum if and the column name is starting from M2 till M5 value should be greater than comma this is M2 right M2 to M that’s it and we’ll just press enter so it’s showing 17 So based on this formula bar the formula will check the values from M2 till M5 and if the number is greater than five and sum only those number so as I’ve mentioned the value five so it will only calculate it will it will only sum numbers which are greater than five which are 10 and 7 which is equal to 17 so this is how the summ function is used we have come to the end of our video and this was all for the top 10 most important basic Excel functions if you categorize the steps to become a data analyst these are the ones firstly you need to focus on skills followed by that you need to have a proper qualification then test your skills by creating a personal project an individual project followed by that you must focus on building your own portfolio to describe your caliber to your recruiters and then Target to the entry level jobs or internships to get exposure to the real world data problems so these are the five important steps now let’s begin with the step one that is skills so skills are basically categorized into six steps it cleaning data analysis data visualization problem solving soft skills and domain knowledge so these are the tools Excel MySQL our programming language Python programming language some data visualization tools like tblo powerbi and next comes the problem solving so these are basically the soft skill Parts problem solving skills domain knowledge the domain in which you’re working maybe a Pharma domain maybe a banking sector maybe automobile domain Etc and lastly you need to be a good team player so that you can actively work along with the team and solve the problem collaboratively now let’s move ahead and discuss each and every one of these in a bit more detail starting with Microsoft Excel while Advanced tools are prevalent Proficiency in Excel remains vital for data analyst Excel versatility in data manipulation visualization and modeling is unmatched it serves as a foundational tool for initial data exploration and basic analysis data management database management skill is indispensable for data analyst as data volume saw efficient management and retrieval from ddb is critical Proficiency in ddb systems and querying languages like SQL ensures analyst can access and manipulate data seamlessly followed by that we have statistical analysis statistical analysis allow analysts to uncover hidden Trends patterns and Corr relationships within data facilitating evidence-based decision making it empowers analyst to identify the significance of findings validate hypothesis and make reliable predictions next after that we have have programming languages Proficiency in programming languages like python is essential for data analys these languages enable data manipulation Advanced statistical analysis and machine learning implementations next comes data storytelling or also known as data visualizations data storytelling skill is Paramon for data analy data storytelling Bridges the gap between data analysis and actionable insights ensuring that the value of data is fully realized in a world data driven communication is Central to business success data visualization skill is a CornerStore for data analyst as data complexity grows the ability to present insights clearly and persuasively is Paramount next is managing your customers and problem solving managing all your customers data and companies relationships is Paramount strong problem solving skills are important for data analyst with complex data challenges and evolving analytical methodologies analyst must excel in identifying issues formulating hypothesis and devising innovative solutions in addition to the technical skills data analyst in 2025 will require strong soft skills to excel in their roles here are the top FES data analyst must effectively communicate their findings to both Technical and non-technical stakeholders this includes presenting complex data in a clear and understandable manner next soft skill as teamwork and collaboration data analysts often work with multidisciplinary teams alongside data scientists data Engineers business professionals collaborative skills are essential for sharing insights brainstorming Solutions and working cohesively towards common goals and last but not least domain knowledge knowledge on domain in which you’re currently working is really important it might be a phical domain it can be an automobile domain it can be banking sector and much more unless you have a basic foundational domain knowledge you cannot continue in that domain with accurate results now the next step which was about the qualification to become a data analyst Master’s courses online courses and boot camps provide strong structured learning that helps you gain in-depth knowledge and specialized skills in data analysis masters programs offer comprehensive academically reest training and often include research projects making sure you’re highly competitive in the job maret Market online courses allow flexibility to learn at your own pace while covering essential topics and boot gaps offer immersive Hands-On training in a short period focusing on practical skills all three parts enhance your credibility keeping you updated on industry Trends and make you more attractive to potential employers if you are looking for a well curated allrounder then we have got you covered simply learn offers a wide range of courses on data science and data analytics starting from Master’s professional certifications to postgraduation and boot camps from globally reputed and recognized universities for more details check out the links in the description box below and comment section now proceeding ahead we have the projects for data analyst data analyist projects demonstrate practical skills in data cleaning visualization and Analysis they help build a portfolio showcasing your expertise and problem solving abilities projects provide hands-on experience Bridging the Gap between Theory and real world application they show domain knowledge making you more appealing to employees in specific Industries projects enhance your confidence and prepare you to discuss real world challenges in interviews proceeding ahead the next step is about the portfolio for data analysts a portfolio is a testament that demonstrates your skill and expertise through real world projects showcasing your ability to analyze and interpret data effectively it provides tangible proof of your capabilities making you stand out to the employers additionally it highlights your domain knowledge and problem solving skills giving you a Competitive Edge during job applications and interviews last but not the least data analyst internships internships provide hands-on experience with real world D sets tools and workflows Bridging the Gap between Theory knowledge and practical application they offer exposure to Industry practices helping you understand how data is used to drive decisions internships also build you Professional Network enhance your resume and improve chances of securing a full-time data analy role so now we enter the demo inventory in Microsoft Excel so we will be using Microsoft Excel to create a sheet of the employees in a company so basically an employee in a company has employee ID name and designation salary etc etc so we will be trying to create the same table using Microsoft Excel but before that let us understand the fundamentals of Microsoft Excel through the the Practical demo first so I have started my Microsoft Excel and this is how the homepage of Microsoft Excel looks like so you have a blank workbook over here if you want to create a new workbook you can select new Option so Excel will provide you with various variety of sheets you can see money in Excel adjustable meeting agenda streaming showers small business cash flow and many more if you’re not able to find what you’re looking for then you always have an option of selecting the particular type of sheet what you’re looking for so you have various options if it’s business if it’s personal if it’s planners and trackers list budgets charts etc etc so let’s imagine that you wanted something from business so just by clicking at business option the Excel will load a variety of sheets related to business options so this might take a while so you can see that the Excel is loading few types of sheets so you can see that Excel has provided us with some online varieties of sheets for example any calendar business expenses Channel marketing budget budget sumary report blue product list etc etc you can see construction proposal Eno you name it Excel has got it so Excel will provide you with some variety of options based on your requirement now for this session let’s get started with your blank workbook which looks something like this since this tutorial is based on the fundamentals we’ll go with the blank workbook now over here you can see the toolbar that we discussed earlier that has the file home insert draw page layout formulas data review View and help so this is the toolbar and under the toolbar I have selected home and you can see this is the particular ribbon what we discussed about this particular ribbon belongs to homepage and when you select file option you’ll get back to home and if you select insert option you’ll have another different ribbon with different groups etc etc so every particular tool has different ribbons in them and remember the extra options that we discussed when you press over this Arrow key this is it so when you press the arrow key you’ll have few more settings which cannot be fit in this this particular section of groups so you can have some variety of options over here of changing the font changing some effects to the text font size and font style etc etc apart from this we have also discussed about the cells in every sheet so this particular cell has an address so you can see the address over here which is B3 so B happens to be the column name and three happens to be the row name now we also had a discussion about the sheet tracker right now we just have one sheet if you want multiple sheets you can just feel free to select on the plus option which will always create you some extra sheets and you can navigate through sheets just by pressing on the sheet name and when you get back to the bottom right corner you have an option of increasing and decreasing the cell size or the sheet size now let’s keep it default with 100% now these are the few fundamentals that you need to keep in mind before getting started with Microsoft Excel now that we know the fundamentals of Microsoft Excel let’s get started with a practical session which is about the employees details in a company now let’s select this particular cell and let’s type in employ details yeah we have the cell now an employee details table will have the information related to employees so the information will be about about name it will be about employee number it will be the designation and maybe salary and maybe blood group as well and uh let’s take another one which is phone number yeah so so far so good and uh you can see that we have some problem with this particular column the designation the name uh the the name of the designation is practically good but it is not visible so when you uh when you’re not on that particular cell you can see that the name is incomplete over here so you can always fix that you can just you know manually change the size of the row or cell or you also feel free to you know double click on that cell which will automatically you know set the size of that particular cell and same goes to the blood group just let’s try to double click on that and same goes to phone number great so you can see that the employees details are just confined to the first two cells it’s supposed to be somewhere in the middle right so no problem we can do that as well we can select all the cells and we have an option of merging them you can just select this one which will help you with merging and centering that particular data to the center part so that’s how we do it now let’s get started by adding the names of the employees uh let’s add the names Joe John uh Mary Mark Susan and then Jennifer let’s type in Mike let’s type in Tim Jeff Jeffrey yeah we have a couple of employees now let’s type in the employee numbers yeah we have the employee numbers now let’s type in the designation okay let’s choose uh Joe to be the CEO of the company and John as the software developer and Mary as tester Mark and finance Susan also in finance and Jennifer in testing and mik in uh marketing same goes for Tim and again Jeffrey into software development and uh Ming into testing again now let’s click on C so that it gets you know resized according to the length of the text now it’s done so let’s get into salary uh $10,000 and $115,000 $119,000 let’s increase the salary of a CEO so let’s keep it $1 lakh and finance again 20,000 so yeah the salaries are allocated again the blood group yeah we have the blood groups now let’s type in the mobile numbers for yeah now we have typed in some random mobile numbers as well so uh yeah this is how you can add in some data into your table and now let us imagine that you forgot to add or remove a column so let’s imagine that we wanted to add a serial number as well but somehow we forgot to add it now you can always add a new row or column for example here we wanted to add a new column so we just have to right click on a select the insert option here we have a new row now now let’s type in serial number and let’s type one now can you see the small box option over here if you just drag it you can you know copy paste all those over here and now let’s right click and fill series now we have the employee number starting from 1 to 10 that’s how you do it and apart from this you you can also uh you know change the font of the entire row you can change the font to say uh aroni and you can always also change the font and same goes to the employee table you can select it bold it and you can also increase the size and again select a color for the text maybe a different color green would be better and and uh you can also select the entire cells and align them to the center looks more good and you can select or double click the row names so you’ll have the proper spacing between all the rows and columns so we have double click on the column right yeah so basically that’s how you make things happen now let’s save this so I’d like to go to the save option and uh EMP data let me save this in my local location and just save it’s done so that’s how you work on your Excel file with some basic data and to learn more don’t forget to get subscribed to Simply learns YouTube channel and don’t forget to hit that Bell icon to stay updated so you are the first person to get an update on any technology not just Excel now let’s get back to the Practical mode and now we are on our Excel spreadsheet so here here we have some text boxes on the column A and on column B we have lower and upper okay so um we have a mix of uh uppercase and lower case in our text so first we’ll try to convert them to lower case and then we’ll try to convert the entire text into uppercase so for that you need to press on equal symbol and select the lower formula tap and you selected the function and now click on the cell A3 that’s your select press and press on okay now all the letters are converted to low case so we had J in the uppercase which is converted to low case now all you can do is drag the formula to all the cells and it will be applied to all the cells right now let’s similarly try to convert into uppercase and tab space and there you go you have selected the formula now the cell address press enter and drag the same address I mean drag the same formula to all the cell addresses and there you go all the cells are converted to uppercase now that’s how you convert the cells or data in Excel to uppercase or lowercase in Excel now we are on the Excel spreadsheet now here we have some sales data based on a store now what we going to do is we going to add some multiple rows or a single Row in this particular data so here you can see the region wise category Wise statewise subcategory Wise and the sales and quantities right now let us imagine that you must have to add a sale data here right so let us imagine that on the row 7th you were supposed to add some detail for example some technology related sale happened on that day and you missed out to add it right so how do you create a new rule now so for example if multiple sales happened on the same day like 3 to four and you missed out them so how do I add three to four rows in between the ex testing data right so don’t worry about it it’s completely simple all you have to do is select the row and right click and press on the option insert now this will insert one single row if you wanted to add one single uh row or you know toule and if you wanted to add multiple rows for example you wanted to add three rows Al together then it’s also simple all you have to do is select three rows and right click and press on insert and then you have three rows so there you go so that’s how you add multiple rows in a Excel spreadsheet or that’s how you add one single Row in an Excel spreadsheet now we have started our Excel worksheet and here you can see we have employee details on our worksheet now all of a sudden uh you wanted to add employee second name as well or a last name right so you are a beginner and you don’t know how to add a column or you know you might be a little ambiguous right so instead of using this simple step you might end up creating a whole new sheet or maybe other approaches as well which are timec consuming so let me give you this simple step all you have to do is select the column so here you wanted to add a last name right so B is your column with the first name and here you wanted to add a second name or the last name which is in between the columns B and C so remember that Excel adds a column always towards the left side of a column so here if I if you wanted to add a column next to V then you want to select the column C and right click and select the insert option so that it can be added in between B and C which is adjacent to B so let’s look at it right so you have added the next uh column or a new column right next to the column V and it’s in between the designation and employee name now you can add your you know second name so that’s how you create a new column in Excel now in case if you wanted to you know create multiple columns all together so uh here we have or here we needed a second name also here you have a designation and you wanted to add another column which you know assigns managers to the employees right so let’s delete it uh Delete the second column and let me explain you how what I want to do so here you wanted to add the second name of the employee and also the manager of these employees right so here you wanted to add two columns Al together right uh and now you might be wondering could I add two columns together or will it work only for the one column thing right no it it works for two columns and also more than two columns all together so you wanted to add two columns here right so select the two columns you you can select or you can click on the column see and hold that left key and drag it along the right side Okay so here I have selected two columns and here it’s three right so you can see here uh it’s 1 48 5 6 7 6 are cross 3C that means you have selected three columns if I navigate on the four you can see we have selected four columns right now right click and select on insert and you’ll have four columns in between employee name and designation now you might have to add your second name man man name manager employee ID you know and the department everything or you can add anything or any number of columns in between the existing columns now we are on our Excel spreadsheet now you might want to you know add uh some extension to your data since we have sales here you might want to add the dollar logo right so if you have like five to 10 rows then it’s easy but if you have multiple rows then it might be a little time consuming in such scenarios you might want to select the entire column right so how do you do that in a shortest method so there are multiple methods to do it so the easiest one the first and the easiest one is clicking on the column number or column name so here we have the column name that is e and if you click on it the entire column is selected and here we have numbers so let us imagine that you wanted to select the row number two click on number two and you have all the rows I mean the entire row selected all the cells are selected right so this is one way and again there is another way where you can just you know uh click on the first table header and press control spacebar so the combination key is control spacebar for columns and if you want to select the entire row it is shift space bar so this is the method two and another method so you might have a doubt here right so when you click on the column name or when you click on the row number the entire row or entire column gets selected right and you wanted to select only the cells which have data so we have another way for that so you can select the cell now hold the control and shift key and press the lower Arrow key to select the entire column with cells having data so there you go you have selected all the cells in one single column with only data and all the cells which do not have data are not selected same applies to rows as well control shift right arrow key or left Arrow key based on your rows and all the rows with data will be selected I mean all the celles in a row are selected not the entire row and U coming back to the first question that is adding the dollar symbol so here we have General so go to the data type and add currency so you have rupees here so currently we are in India so we have rupees now what is the advantage of selecting the entire row when you don’t have any data in the 57th row or the next cell right I’ll show you now let us add the number 100 and press enter there you go let us add another number like th000 and press enter so the data formatting is automatically applied to all the cells so most of the time your cells or the data sheet will be varying you might have to add or remove elements or numbers right so in such scenarios this will be helpful now comparing two different columns or multiple columns happen Happ s to be an important job when you’re working in data analytics as you have to come up with some decisive decisions based on the data now if you had to do it manually then you might end up taking hours or even days based on the data set you’re working with but if I say that you can do it within minutes then it would be interesting to work with right now we’ll be doing the same now we’ll work on a sample data set right on my screen so here we have column 1 and column two now our job is to compare the column one 1 and column two and come up with the result now the first and the simplest way to do it is use the built-in conditional formatting which comes with Excel by default now all you have to do is select all the data and navigate to home and then the home go to Styles group and select conditional formatting and in conditional formatting you can see highlight cells so in that you can use duplicate values when you click on duplicate values a small pop-up window will come on your screen and here you have an option of choosing whether duplicate or unique so duplicate means you’re comparing the cells and you can see there are some duplicate cells which are present in column one are also present in column two now you can also check how to uh you know find out the unique values which are only present in column 1 but not in column two so you can just press unique and there you go you can find it now you can also try to you know change the color by filling with green for The Unique columns and duplicate cells with color red now this was the first method and you can also work out some different ones like uh you can just directly uh press equal to and select the cell and equals to and the next cell press enter and if there is a match it will give you true if there is no match then it will give you false now you can drag it down and see which all are matching and and which I learn not now you can also make some minute modifications to this so uh if in case you didn’t find the data then you can say not found so for that uh you can try if and inside brackets and then you can give the value as not found if true it’s found first you have to give the True Values so for true you can write it as found and in case if it’s not found then you can write write it down as not found there you go close the bracket press enter and there you go if there is a perfect match you will have found and if there is no match then you’ll get not found now so far we have discussed uh comparing two cells using conditional formatting and also by using equals to operator and trying to uh add some tweaks to the equals to Operator by involving with if operator and apart from those we also have another way to compare two columns in Excel so uh another way is using the lookup functions so we’ll use a simple V lookup function to compare both the cells so for that let’s type in equals to we look up time space and um select the cell you want to check for and then the range of elements you want to uh compare and then I’ll press F4 to log them now uh you want the data from column one and now you want the exact match so zero press enter so there you go so the elements which are present in column 1 and column 2 are been displayed here so but in case if the elements are not found you’ll get an error so let’s look at that simply drag the formula to all the cells and there you go so the elements which are matching will give you the proper result but the ones which are not matching will give you not applicable or error message this can be fixed we can make some minor tweaks to the same formula so uh we can add if error comma simply write an as not found and close the bracket so that should do so we have closed the bracket and press enter there you go now simply drag this and soon you can see the data which is having no match is shown as not found now uh there is another possibility okay let’s get back to another different sheet so this should work so uh we we already have the formula so let’s erase that first I’ll erase the whole data and even this one yeah now we have the clear column for so here you can see we have Ford India and here you can see we have just Ford and similarly here you have Mahendra and Mahendra and here you have Mahendra right so in some situations you might have to compare two different columns but the names might be a little different right so for example uh if you’re working with Oracle and if you’re working with Oracle America and you have in the first case is Oracle and then the second column is Oracle America then those are one and same but you have some minor text changes right just like here you have for India and here you have Ford so what if you had such kind of issues so you can also make some minor tweaks to it so using the same lookup formula you can add equals to we lookup and also the comparision cell which is this one and you’re not you’re not stopping them you’re trying to add the wild card here so the wild card is HRI which means if there is anything like uh if you get the comparison between the first cell and and the second cell which matches and if there is anything extra rather than the actual cell please try to consider it so that’s what we going to do so after entering entering the wild card symbol you’ll select the range of columns which you have to uh you know fetch the data from and then try to fix them using function key F4 you’re looking from the column number one and then you’re finding the exact match now you can close it and ENT now let’s try to drag it there you go so you have the data Mahindra and Mahindra hindai India Honda India so that’s how you try to use uh comparing cells in Excel now you can see I’m on my Excel worksheet and I have one table on my worksheet right now so you can you know transposing or you know converting rows to columns is a very simple task I’ll explain you in two different ways so the first way select all the sales that you have on your sheet and press on contrl C so it will give you the copy option and then you can also paste in the same sheet or if you want you can go to a whole new sheet and select the paste option over here but before selecting the actual paste option you can see a small drop- down icon click on that and navigate to paste special and in this you can see transpose option click on that or you know make sure that it’s checked and press okay now you can see all your columns are been converted into rows and all your rows are converted into columns so this was the first way and uh what is the second way you ask me so it’s really simple select one cell somewhere and you know you can or let’s create a new sheet as well so here you can you know write down the formula as transpose and select the array which you want to transpose so this is the array so transpose is basically an array function in Excel so once you are done with you know selecting your data press enter there you go so all your rows are now columns and all your columns are now the rows now on my spreadsheet you can see some data which is the sales data of quarterly basis and the four different zones that is northwest southeast so you can you know make things look better by hiding these columns and if you hide these columns it’s critical for the end user to identify which columns are exactly hidden right to make things a little easier for your end user you can select the columns and then go to the data option and in the data option click on the group option so this will group these columns and you will have the symbol of minimization which will help you you hide these columns automatically and unhide these columns automatically now let’s also try to group GH i j columns and there you go now similarly let’s also try to group the rows of North and Rose of West then you have the rose of South and row of East right now all these rows and columns are been grouped now all you can do is just click on the minimize button and those will be minimized similarly the rules right and to unhide them all you can do is click on the plus icon and they will be unhidden you can also try to ungroup the columns and rows all you have to do is select the columns and click on the ungroup option similarly the rows as well and that’s how you can group and ungroup rows and columns in Exel now we are on our spreadsheet and you can see I have some sample data on my spreadsheet and it does have some blank rows now how do I eliminate these blank rows so one step is you can you know select the entire row and try to right click and delete it again this sort of method might take us to get rid of all the blank rows and most of the time you might end up even selecting the rows with data in it right so if I want to select multiple rows then I’ll be selecting some rows with data as well which might be a little Troublesome in the future now let’s try to use the easier way now for that go to the find and select option select the go to special option and in that select the blanks option and press okay now hold control key and press minus key and you will have an option of deleting the cells now select the entire row now we are selecting the entire rows with blank data and press okay and the cells will be shifted now this was simple but let me show you another example so even here we have some blank rows but in between those blank rows for example will take cell 17 or cell d17 or row 17 we have some data testing and here in the employee ID we have some data in a26 and again in b39 we have some data that is first name right in such scenarios how do I eliminate data so we have a simple Logic for it so all we need to do is add another row which is to count the number of cells in the row now that can be count now here equals to the function is Count a let Tab and the cells will be from A2 to G2 press enter now you have seven so now we are basically calculating the number of cells so if you apply the same to the entire um data set let’s drag it and there you go so there are certain places where you find zero which are basically the empty columns right now let’s do some formatting if you need let’s try to add filters crl a contr d and now table will have filters and headers now here you can see the drop- down icon click on that now instead of select all what you can do is Select zero now you will be highlighted with all the rows which do not have any data select all those and press on delete or what you can do is the same way control minus there you go now clear the filter now all you have is the data without any kind of blank rows so all the rows that had some amount of data are retrained and all the data or all the cells or all the rows which do not have any data which are completely blank they have been eliminated so here you can see columns and rows so especially the fourth row and the column C right so uh currently we have data on our uh spreadsheet and it is exceeding the number of rows currently visible on our screen right so when you scroll down you can see that the header or the headers of all the columns are getting vanished or scrolled up along with the data right and you might want to keep that so that you’ll have reference to every single aspect of or every single cell data you’re referring to right so for that reason you might want to freeze this and also you might have a doubt what if uh even even I had this data in the column section right and what if I had to freeze that and you know when when you scroll this and keep that data uh set on the column as it is right so we can also do that let’s look at both of them so there is a simple process to do that all you have to do is select the cell or for which you want to uh you know set the or freeze the column adjacent to it or the row adjacent to it right so currently you want to let me expand it so currently I want to freeze this particular row right and this particular column so the cell I would be selecting is the first cell here that is D5 right so when I select this particular cell I can be able to freeze this particular row and this particular column so now you need to navigate to the toolbar and then select the view menu and inside view option you can see there is an option of free pains right so click on the drop down so you can see three different options as we discussed earlier we can either choose to freeze an entire row that is the second option or you can freeze the entire column that is the third option or you can freeze both that is the first option right freezing rows and columns so let’s select this one and see if our rows and columns are frozen or not right so now let’s scroll down and you can see our row is successfully Frozen there right and similarly scroll towards right and you can see our column has been successfully Frozen right so this is how you freeze rows and columns or you can freeze panes in Excel now here on my sheet you can see we have some numbers that is $10 or 10 .20 $22 $ 53.1 two $12 and $110 now the task is to convert these numbers into word format that is representing the numbers alphabetically right this might be a little curious and we also might end up searching for a predefined method which is not readily available in Excel but there is a way where you can create a function using Excel VBA or macros now again creating a macro is completely complex now I identifying the trouble Microsoft has already come up with a readily available macro code on their official website which will support the various versions of Microsoft Excel starting from Microsoft Excel 365 to all the way up to the older Excel version that is 2010 now you can scroll down and use this particular macro function which Microsoft has readily made us available and use this to create our own macro in Excel VBA now let’s get back to Excel and you can create a macro using the developer option now developer options will not be readily available in Microsoft Excel to activate developer option go to home and go to the options menu in the last and here go to customize ribbon and here in the second drop- down menu you can see developer option by default it will not be checked click on the check option to activate developer options and and press okay now you have your developer options ready click on developer options click on Visual Basic and here to insert a new macro click on the insert button click on module and a new macro window is readily available for you now control V to paste your code and your function is with the name number to text right now close the macro and here your function will be ready available equals to and the function name was number to text press tab to select the function and select the cell where you have the data close the function press enter now drag the same formula across all the cells and here you have the text converted from numbers so here we have $10.20 and you have $102 $22 and no cense $53 and2 $12 and no cents and $110 with no sense now that’s how you can convert numbers to words using the macro which is predefined for excel in the Microsoft website so for example uh let us imagine that you wanted to create one single cell for all the employees in your company which stes the first name second name or last name including the email IDs at one place right if you want to do so like let us let us let me show you how it looks like so let me copy this here then that’s the first name now it’s your second name along with the email ID now let us imagine that you wanted to store all the employees in the same way right so you can do this by using combine operation in Excel now all it takes is just a simple formula that is equals to and the first cell address that is A2 and you can use an ENT sign now you can use the next cell reference that as B2 and then colon and the last C address that is E2 now press enter so when you press enter or tab you will get this particular answer but this data is completely uh in a state where you can’t understand it right the first name last name and the email ID has got combined all together so you want to use some spaces right so all you need to do is just make a little modification to this particular existing data so you can press another ENT and include spaces and then you can separate the email ID and the name with some colon so now we have separated the first first two cells that is the first name and the last name with the space using the ense and then again we have used an ampers to include the colon and see that we have also included a space between the colon marks that it gets separated cleanly and now let’s try to press Tab and see the result so there you go the first name and the second name is separated by space and the email ID and the complete name is separated by a colon now you can use this to all the cells and there you go all the first name last name and the email IDs have been printed here so that’s how you use the combin operation in Excel and you can see on my screen we have the sales data of one single employee in the month of January okay so this is the sales data of John in January sales now let us imagine that we have this same person’s or same John’s data of sales of all the three months okay so here we have the sales data of John for January February and March and you wanted to Club all these three months and write it down as quarter 1 right so let us try doing it now you can see that uh only January is replaced as quarter 1 and February and March are still uh remaining the same so let us try to delete them so we have deleted them and I think now it looks like uh this is a sales data of one single quarter that is quarter 1 but again quarter one is under column B and anyone who looks at it feels like a this data is of January February and March and all the quarter 1 details are only present in V column but C and D are completely you know different from this one right so you don’t want that kind of an on your uh you know the one who presenting to you so in those situations you want to merge all these three columns and make sure that the quarter one is present somewhere in between the cells so that all the data looks relevant and uh it it defines that these data elements belong to Coto one right so there is an option in Excel to do that so select all the cells that you want to merge and going to home button and in home button get into the alignment group and select merge and center so you can do it directly also you have a couple of more options that is merge and center merge across merge cells and unmerge so we’ll go through it one by one so first we’ll go through merge and center so there you go the cells Cod merged and it has been aligned into center now again if your manager or anyone Superior to you wants this to be you know independent like they want it to be in terms of January February and March so that they can have access to it in month-wise so you want to unmar it back right how to do that now so the same menu and select unmerge and there you go you have your data back in the original form again all you have to do is you know rename a few celles there you go and you have it back right so in a few situations okay you can also add the pters so that it looks more relevant now um you know let’s try to merge this again and merg in Center yeah now again uh let us imagine that you are going through an a pral cycle and you also wanted to add some comments about the sales happening in your company uh regarding uh all the employees right so also we have that here now let me expand a little bit Yeah there so again let’s margin center now uh you wanted to you know uh you wanted to take a review of your level one level two and level three managers so you have all the three um rows here but you can see the cells are not merged and uh you wanted to merge every single row so merging and centering everything rowwise would be a little time consuming I mean three columns is okay but what if you had like uh you had to take a review from a couple of more managers like 5 to 10 right or imagine you wanted to write some random comments apart from review you wanted to write some random comments based on month-wise uh sales or something like that so it would be timec consuming right so apart from that you have another option where you you

    know select all the sales and come over here and select the option of merge across and all the sales will be merged all together at once and you will be having those individual rows where you can write down your reviews like review one and so on review two right so this is how the option of merge across works and we already have been through the merge and center and merge is completely similar to merge and center so this is how merge and center merge cells and uh merge across and unmerge cells in Excel now there are multiple ways to add dat to Excel spreadsheet for example equals to today this is a short cut method and you can enter done you will have today state but again there is some problem with this kind of approach so today’s date is 4th of September 2022 now if you open the same spreadsheet after today that is tomorrow or day after tomorrow then you will have a concurrent date that is 5th of September or 6th of September right the date will always vary or get updated according to time now you don’t want to change that that or you don’t want that to be happening with employee state of joining right it should be stagnant it should not vary right so there is another shortcut way apart from using the datetime functions there is a keyboard shortcut where you can add date to excel that is control semicolon so this is the shortcut way to add date to your Excel spreadsheet and this particular date will not change this will remain the same constant date throughout the time now there is a chance where you might also have to add time to your spreadsheet that is date of joining and time of joining right so there is another simple shortcut where you can add time by just pressing three keys that is holding control holding shift and pressing semicolon then you’ll have the time so this is how you can add date to your Excel spreadsheet and add time to your Excel spreadsheet and this date and time will remain constant it will not vary like the 2D function that we have used here so you can see on my screen we are on the Excel spreadsheet and on my spreadsheet you can see dates in column A and different formats in column C D E and F respectively Now by default the data type or the data present in the a column will be considered as general and there are a few situations where even if you try to format the date in your format like you know the customized format you may not be able to do it because it’s still under the general data type so for that for being on the safer side what we will do is select the data and navigate to the data toolbar and in that you have the text to columns option press that and remember to have the delimited icon you know you might be having the dash as your icon or slash as your icon for the delimited thing and select that and next next and here you can see right it’s in general mode so you can press on date and DMI will be a date format that is date month and year now click on finish so it’s been formatted now now let’s copy the same data and paste it in all the four formats so that we’ll see the different formats that we can change to or customize to right so there you go let’s expand the columns a little bit now we have day month and year so now you might want to change a couple of things right for example you wanted to change okay let’s see the menu so the way is to select the sales and right click and select the format sales option and in here you can see the date function and in here you can see different types of modifications that you can do to your data right so you might want to also change your dates based on the location so right now we are in India and imagine if you wanted to you know change something based on us or if you if you’re having your client in us and he wants the dates in US format it you can also change that so here you can see English United States and press on that press okay and you can see it’s been changed to us format where you have years in the first place months in the second place and dates in the last place right and now let us imagine that uh your client does not want the dates all he needs is just months right you can also do that get back to the format sales option go to custom and here instead of uh you know uh month date and year all you can do is just write down month and year and select okay so you’ll have only month and year not the dates now your client is being a little more you know he wants a little more detail and he wants you to add a lot of granularity to your data which includes day time Etc everything right so you can also do that so the same way go to format sales custom and here I think it’s in date itself so let’s navigate a little yeah here so you have the day date month and year press okay so there you go right so that’s how you can do it and now let’s try a little bit more if we have a few more options pending so go to format cells and custom and you also have timings so so here you have date month year and timings press okay and there you have all the timings as well so since these dates are you know I created them just a few moments ago to keep it simple so I did not have to add the diamonds here so if you are Keen about adding the timings you can also do that for example if you wanted to add the timings of the employees logged in and logged out you can also do that using the same format and you can see now we are on our Excel spreadsheet and you can see we have a column of date and on the column B we have age now how do you calculate the age using the date of birth so this is completely simple all you have to use is the dated IF function or if you also call it as date diff function based on your choice we also have a specific entire tutorial based on Dat diff in Excel for further information that video will be linked in the description box below you can go through it or you can use the end screens and ey Cuts linked to this video to get back to the tutorial based on date diff in x now let’s calculate the age of these dates or these people with these date of births so all you have to do is type equals to and type dated F and open a bracket we need three parameters here that is the date of birth that is the first parameter now the second one is the today function because you’re calculating the age of these people as per today right so we have a today function in Excel press on Tab to select it and give a close bracket now comma and if you wanted to calculate the age in terms of days you have to um you know give d as the input yeah here it should be D as the input and don’t forget to use the double codes here and if you use the single codes there might be an error so use double codes if you want it in the day SP you can give days and if you want it in months you can give months for now we want in ear so we will be giving y as the third parameter and press enter there you go you have the age here in 26 years you can drag the same function and you have the ages here as you wanted right so this is how you calculate age in Excel and on my Excel spreadsheet let me expand it we have two different timings that is in and out so here we are considering a scenario where we are calculating the total number of us worked by an employee in an organization so the in time will be the time where the employee logged in and the out time will be the time when the employee logged out so here we have made some minor cell formatting so when you right click on any of the cell you can get back to the formatting sales option and in that option we have selected the AM PM so there is also another option of choosing the 24hour timing that is 1320 uh 13:30 so this represents the 24-hour format but since we are you know working on Office timings let’s keep it as a PM okay so there you go so we have our uh timings of in and out now to calculate the time difference between the in time and outtime so this particular calculation is completely simple all you have to do is press equals to select outtime minus select in time so there you go press enter and you have the total number of asks the m has been working in your organization so here also we have made some minor cell formatting so usually it comes out as Am Pm right but instead of that we have made a minor cell formatting we have navigated into time and we have selected the 24hour formatting so that we have just hours in our output so that’s how we have calculated the time difference between two in times and out times of an employee using Excel now Dax and Excel is a sophisticated formulas type language that comes in handy while working with relational data and extracting information via Dynamic aggregation functions Dax in Excel stands for data analysis Expressions Dax functions are completely familiar to the general and default functions that are available in Excel now Dax allows users to perform slightly escalated and advanced and custom calculations upon various data types like character data date time time intelligence functions and many more and in Dax there are a variety of functions such as Dax table valued functions Dax filter functions Dax aggregation functions time intelligence functions date and time information time logical parent and many more now in today’s session we will look into one such type of Dax function in Excel that is Dax date and time function in Excel now without further Ado let’s get into practical mode and we have started our Microsoft X Excel inside Excel we have opened a new blank worksheet Now navigate to data option in the toolbar and in that go to the power pivot window now we have entered the power pivot window and inside that you can see the option of get external data click on that and you can see a variety of options that is you can get the external data from a database you can get from web servers that is from data service and you have also other forms of data sources and existing connections for now we will take other forms of data that is a Microsoft Excel data which is in our local system select next now provide the location of your folder just browse and we will select Dax employee data open and there you go we have our data connected to our powerpint window click on next and there you go you have the selected table and views click finish and you can see the data getting imported into your power window there you go by default if you want to add any tax related functions into your powerp window you’ll have an external added column over here which looks something like this if there is not an external column you can also add the column it’s not a big trouble now we have some data here that is the employee ID employee name designation of the employee Department of the employee salary and joining date Etc right let’s expand this View and see what’s what’s more in it so this column is the date of birth and this column is the employ joining date right now we have so much of data now let us imagine that we want to calculate the retirement age of all the employees so now let’s rename the column to retirement date so we have our new column that is retirement date now we will make use of Dax datetime functions to calculate the age and find out the exact retirement date of that particular employee so let us imagine that every employee will retire after the age of 65 so we have our date of birth of every employee and the joining date of every employee now we’ll come up with a da de time function and calculate the age and then we will find the retirement date of that employee now every Dax function starts with the same method or the same way as the simple Excel default method starts that is by starting with an equals to and then we will find eate eate and and now our first parameter that is the start date that is the employee uh the date where the employee joined the organization so our sheet name is sheet one now our joining date is in the F7 column select F7 press tab to select it and comma Now how many months so each and every year will have 12 months so 12 months into the age that is 65 Star 65 and now you can close the uh function and press enter and then you’ll find the retirement date of that particular employee so there you go so we have the employee dates that is Employee Retirement dates for all the employees so the first one that is Jack has the retirement date of 19 February 2085 and similarly Jennifer has her retirement date at 15th of August 2084 so that’s how we use the Dax functions in Excel so this is one of the several Dax functions in Excel that is Dax date and time functions in Excel so you can see that I’ve got some data which is first name of the employee last name of the employee and phone number and let us assume that uh your manager has given you a task to identify all those uh employees which did not give an phone number so you wanted to identify them using a check box right like you will be providing a check for all the employees that we have received the phone numbers and you will not be checking the employees which did not give the phone number right so let’s try to do that so to include a checkbox you might want to get into the developer option but for few reasons Microsoft has disabled developer option by default to enable developer option all you have to do is just navigate anywhere on the ribbon here and right click and then you can see customize the ribbon option when you click into it you will be redirected into the ribbon option and here you can see the developer option on the right side right this one so by default you can see that your developer option is not enabled to enable it you have to check in it and there you go you have it now so press okay and you’ll have the developer option now go to the developer option and select the insert option and there you can see the checkbox now draw the checkbox any I’ll draw it over here and there you go you have the first check box right now if you want you can even customize this so carefully click it you can edit the text and and write it as done okay now you can even copy this to all the cells just select and drag and there you go now you can go ahead and check all the employees which you have received the phone numbers and you can eliminate the ones which did not provide you with the phone numbers that’s how you can do it and there’s also another option where you can use this to include it in a formula since this data is only of 25 rows you can manually check in the data and you can leave the data which is you have not received but what if you you had 2500 right then you might need formula so in such scenarios you can use a formula so how do you use that so for example uh if this particular box is checked then you will get a True Value here and if it’s not check you will have a false value then you can use that Boolean value to include in your formula right sounds interesting now let’s try to implement that you can right click the cell and you can find format control then you can see here you have an option called cell link here select the cell link and press okay so here you can see if you have checked this the value is true and if you uncheck this you can see the value turned false now you can use this particular cell reference for including this in a formula now let us also check how to include checkboxes on Google Sheets in Google sheet it’s really simple all you have to do is get into the insert option and you can see readily available check boox you don’t have to use any developer options over here and there you go you can just check now let us assume that you’re giving an important presentation and you’re also requested by your manager to add the Excel document that you have used to create that particular report which would be something like this when you click on it it will open a new Excel document or Excel Window with all the data in it right so how do we do that let’s look at the demo now let’s have a blank slide and let’s get back to the source where our Excel file is located that is in my documents so we are currently in my documents and we have the Excel file which we used for creating that particular report now if we copy and try to paste it on our Excel document it would look something like this and clearly it’s not visible and it’s not accessible right so instead of this there is another way where you can directly add the Excel document in the form of logo which is a rather simple idea all you have to do is just click on the insert option in the toolbar then navigate to the text group and here you will find the option called object click on the object and it will give you a new window where you can identify which type of document or object you want to add either create or create from file so we have the file in our local system so we’ll choose create from file browse and from browse we will go into the documents section where we have our Excel file click on the Excel file and click on okay now if you want the data or Excel file to be displayed in the form of an icon then click on the display as icon section and here you can change the icon so you can choose any one of the given options let’s choose this particular one and and click on okay another okay and there you go the icon will be now displayed on your Microsoft PowerPoint PPD which is completely clickable as the previous one whenever you click on this the Excel document will be opened so that’s how you add Excel document into Microsoft PowerPoint now we are on our Excel spreadsheet and here I’m trying to insert some images onto my spreadsheet so here we have company logos and Company brand names so since we already have the brand names what we need to add here is the brand logos here so how do I add such kind of an image into an Excel spreadsheet that you might be thinking that Excel spreadsheets are just for data not multimedia image of files right so no it’s not it’s completely wrong you can also add your images onto your Excel spreadsheets it’s completely simple so click anywhere on the Excel spreadsheet navigate to the insert button select the illustrations option from the addin group click on that and you know in the drop down you can see the pictures icon or the pictures uh button click on that and select from this device or if you have an online stock image you can take that or you can also check out from online pictures for now I’ll select the device this device and here I have some images the images what we need are simply learn and simply code logos press okay or insert so so here you have them now you can also adjust the size of the image using the small buttons here PR it over here and done so you might be wondering is it all done is there anything more to do uh or just adding an imag is done no most of the times when you create or you work with Excel spreadsheet you will not just you know add some data and add a relevant image to it most of the time you add some graphs or you most of the time you’ll add some charts uh to showcase the data dashboard right so it will be something like this so this is how you add the image or graph onto your uh uh dashboard and sometimes you might not have enough play to uh display your entire dashboard during those situations what you do is you try to hide the row or column so now let’s try to hide this particular row let us imagine that you wanted to show only uh two uh charts and there is no space for your third chart in such scenarios you might want to hide or remove that third chart okay so let’s try to hide it and see if gets if it gets hidden or not when you try to hide it you can see the column A has no data and uh the row number four has been hidden so we have three and five but what about the image it is still present right so if you want to make sure that the row also hides the image then you might have to do a little formatting with it so select the image go to picture format or you can also right click and directly navigate to picture format this opens up a set of options and here you might want to go to the uh third uh one which is picture format third and select the properties and here select the option of move and size with cells so when you select this when you try to hide this it will also I mean the graph of the image will also hide along with the R so that’s how you do it so how to insert PDF in Excel now there are multiple ways to insert a PDF into Excel so we will go through a couple of ways first is here let us imagine that we wanted to insert some PowerPoint presentation in PDF format into our Excel sheet so that it belongs to excel Basics now click on the cell where you want to add the PDF or just click any cell in the Excel spreadsheet then go to the option of insert in the toolbar then navigate to text option click on it and select object and here you can select create from file and then you can can browse and here you have your Excel Basics PDF in my documents click on that select insert so here you can just press okay and the PDF will be inserted now you can minimize the size of PDF to fit the cell there you go the PDF code successfully inserted into your Excel spreadsheet now let’s discuss about the second way now the second way is completely same the only difference is you can add some logo to your PDF so you can see it has been added in direct form of a PDF like what’s the first slide of the PPD but instead of that if you wanted to highlight it as an icon then you can select the display as icon button here and press okay now you don’t have to reformat the size of your PPD or PDF all you have to do is add the logo to your PDF and it will also include the title so this is another way of adding you know PDF to your Excel sheet now another way so a few times you don’t want the first page of the uh PDF to be visible or any kind of logo of that PDF reader to be visible on your spreadsheet sometimes all you want is some dedicated image or a dedicated logo right so this time let’s try to add a dedicated image to our PDF so go to insert again here you can see the illustrations option click on picture and from this Dev device so I will select the simply larns logo and now you have to resize the logo a little bit to fit the cell there you go you can uh you know click on the right click button and and select the option of Link now um select the existing file or web page option so if you are interested to learn more on Hy Link in Excel we have a dedicated tutorial on that please go through hyperlinks in Excel by simply learn which will explain you everything about hyperlinks in Excel how to insert them different types of documents to be inserted and how it works in real time for now let’s try to insert the existing web page and go to documents navigate to documents and inside this you have the Excel basic PDF click on okay and it has been added to your image so when you click on it it will open the PDF document attached to it so when you hover over to the cell you can see it will ask you a permission do you want to continue or not just click on this ignore the warning so you can see the PDF has been successfully open convert PDF into Excel now to do this let’s get back to the Practical mode and try to convert some PDF data into Excel now you can see the PDF document which I’m looking to convert from PD PDF to excel so this particular PDF document has a table so this is the statewise GST collections that happened during March of 2020 now we need to convert this tabula data from PDF format to excel format now we are on the Microsoft Excel now to initiate the process we might want to choose a blank workbook for this so now we are on the blank workbook and the complete sheet is empty now let’s go into the data option in the toolbar and in the data option of the toolbar we have the ribbon of get and transform data in this ribbon you can see get data option select that and you can see various options here from file from database from Azure from other sources Etc now we need the first option that is from file and in that we have a drop- down and in that drop down you might to select the PDF if you have a Json file you can also choose from a Json and if you have the data from XML you can choose that and even you can extract the data from text or CSV as well now we need the PDF option so select the PDF Now navigate where your file is existing so my file is on the desktop and now let me select the PDF document select import now Excel will automatically analyze the tab data in the PDF format and give you the results now according to excel there are multiple table formats table one table two so basically this table is one and the same so all the 29 states are fixed in one table itself but since the PDF is divided in sheets this particular document is considering the first table as a separate table and the table which is present in the third page as a separate table so we have table one table to and there are a few more tables which Excel is assuming that it might be table but it’s not and another table which is page number 002 it is a table and another one is right here now there is an option of selecting all the tables all together at once or you can select only one table which you want to select so if you select multiple items Excel will automatically give you an option of choosing your tables you can just take and select the tables you want or you can directly select the table you want on your Excel sheet so right now let’s try to select multiple tables so I’ll be selecting 1 2 4 and five and this particular one is not a table so I’m eliminating that now you might want to choose the load option so Excel is now loading the data now all the data has been successfully loaded now yeah it is giving us a notification that all the data has been successfully loaded and now just right click on it and here choose the option load to and here select table option and if you want the data to be uh you know loaded to a new worksheet you can choose that and select okay now you can see all the data is been successfully loaded in the form of tabula format in the new Excel sheet similarly let’s try to load another page in the table format new worksheet select okay and the third sheet now get ready to impress your boss by converting all the PDF data into Excel sheet just in a matter of few seconds with few steps and there you go all the four tables have been loaded successfully in the form of excel in just a few steps now this is how you convert PDF to excel that is how to add tick mark in Excel now on my spreadsheet you can see some employee data so they’ve been assigned a task and they want to update the task status as done and not done using symbolic statements that is tick mark and into marks now how to do that it is really simple we’ll discuss two different ways the first way is to do it by conditional formatting so click a sell on the task status go to conditional formatting click on the new rule yes of course we can use the icon settings but we want to automate it so click on the new rule right so here instead of two color scale select icon sets now go to the settings here we don’t want the third set sy so select no cell icon and here instead of percentage number formula let’s add just a simple number and the second one as well just a simple number now in the icon settings select tich Mark you have various types of them you know the circular Tech Mark and let’s select the generic tick mark without any outer circles and into here generic X Mark and now let’s set the values when the value is greater than zero it will be green and when the value is less than or equal to zero it will will be X and the reference value is zero and here also the reference value is zero everything is okay click on okay and done and here you cannot see the logos yet we need to add some formula here which is equals to if click on Tab to select the symbol and if Cell address C2 is equals to done then the value should be one else the value should be zero close the bracket enter right now you can see this particular logo here now drag the same formula it’s done now you can see we do have numbers you can eliminate those numbers as well just edit the rules and here you can see show icon only click okay and the numbers will be removed also edit the second one and there you go the numbers are gone and only the symbols remain now what is the second technique the second technique is available in the developer options you can activated by you know uh going into the options here go to file and here go to options and here go to Advanced customized ribon and in here on the right hand side you will get developer options click on the tick box and it will be enabled now go into developer options and click into insert menu click on the form controls or active X controls select the checkbox and now just place it over here you know you can edit this particular checkbox delete everything drag it as checkbox as well now you can drag the same cell across all the functions and you will have the checkbox as well now you can click on it and say if it’s done or not that is how to add Watermark to excel now on my sheet you can see some data so here we wanted to add some Watermark logo to your uh Excel spreadsheet so that it looks genu and work by you right so that’s the overall idea of adding a watermark now go to the insert window and in the insert window go to the text option click on that select header and footer once you select header and footer you can see something on your screen like this so in this box which is highlighted by a cursor is where you’re going to add the picture on my top in the menu bar you can see header and footer elements in this select the picture element and it will navigate to you to a different option window since we are adding the image from our local system select from file or if you are adding from your Bing or any search window you can also add that or one drive as well but now let’s go with from file it will directly navigate you to the pictures folder in your local system so here is my SARS logo click on it and select insert now you can see the images added in this format which is ENT and picture and you might wonder where is the picture it is actually present to view it click on any cell in the spreadsheet and there you go you can see it now you might be wondering the image is a little towards top and abnormally placed right so no worries you can always you know format your picture just press enter and use enter to move your picture towards the center of your data spreadsheet and also there is another option of format picture alog together here you can adjust the height and width of your image by you know increasing and decreasing width or increasing and decreasing height using the arrow marks also works and apart from this there is another option of picture formatting where you can choose your image to be washing out black and white gray scale or anything so currently I’m going with automatic which will automatically adjust the brightness and you know everything for your image it looks perfectly fine so click on okay and there you go right so now this should be fine right now everything is done and everything is automatically saved if you are having that auto save option on your spreadsheet and let’s go back to the normal view from the header and footer view click on The View and go to normal you might be confused right where is the logo it vanish did I had to save it no it’s already saved everything is fine even the image is there but you cannot see it in normal view to see it click on file click on print then you can see the logo it will be present only on the printed Pages not on the normal view okay so that’s how you work with watermarks in Excel and that’s how you can add and customize your watermarks using Excel how to modify or increase cell size in Excel now most of you might probably want to increase just the cell SI you know just to increase the visibility of the text so one of the easiest way is just to press on the plus icon on the bottom right corner of your Excel spreadsheet so this will automatically I Ally increase the text size and you can see the text more clearly now apart from that if it’s just not that what you want to do you can also modify by hovering over to the line between two columns and right click which will give you the option of format cells or column width so either of these options will help you to maximize or minimize your column width similarly you can apply the same idea for your rows as well so you can either format the cells or modify the row height of your cell and this is another way and another simple way is to just hold that line and drag it so it improves your width of the column and another way is to just over onto that line and hold it and drag it to improvise or increase your width of the cell and same applies to your row so this is another way so apart from that there is another way that is format cells from this particular option in the home button you can select this and here you have the row height autofit row height column width autofit column width for example let us imagine that you wanted to you know just by mistake you U you increase the sell size and you wanted to restore it back to the original thing right so you can just click on the autoit height and click on the autoit column width there you go it happens in that way and now coming back to another s way let us imagine that you are given a lot of data and you are told to autoit all the rows and columns now we have already seen the format cell option where you will autof fit all the columns and width of the column and also the height of the rows Etc right but there is also a keyboard shortcut to do this so all you have to do is select a cell and press CR a to select all the cells in the spreadsheet and once all the cells are selected all you have to do is press hold ALT key and then release it then the combination of keys that is h o i so your column width has been now adjusted now the same formula or the same shortcut key with a small modification press and hold and release ALT key now h o and a to adjust the row height so this is the way where you can increase or decrease or modify or autofit all the cells in Excel now you can see that we have the First Column that is a column and the first set of that is text and all the other cells are numbers but as a whole Excel considers the whole column as a general data format so why are we discussing about this so to create a barcode in Excel you have to make sure that all the elements in the column are of Text data type so the first step is to convert this General data type to Text data type so you can do that by going into the data type and selecting text and now all the elements in your first column A are type text now the next step is to convert these elements into barcode now for that let’s create a new column called barcode now our new column that is barcode has been created now the next step would be is to check for the barcode font so the barcode font looks something like this but let me remind you the barcode font in Excel is not available by default don’t worry if it’s not available we can download it from open source so let’s go back to Google so on the Google you want to search uh for barcode font for Excel so the one that I would recommend is three of9 barcode font now you can download this font by just clicking on the download button over here now I’ve already downloaded this font on my local system let’s now try to install that now the three of9 barcode folder will come in the form of zip folder you have to unzip that and after unzipping you’ll get the setup file which looks something like this which is three of9 new select that that and you’ll be having the installer file over here you can install by selecting the install button over here so I have already installed it in my local system so it is readily available for me on my Microsoft Excel now we might want to create the barcode for this particular cell in A2 so let’s create that so to create a barcode we need to write in a formula that is equals to double codes star double codes and ENT symbol address of the cell that is A2 another erson symbol again double codes star symbol and double codes so you can either use this or instead of star you can use brackets as well so let’s try brackets now select enter now this will generate another type of code in your resultant cell now we will change the font for this particular resultant cell and we will get our barcode now you can just drag all the cells so that you can apply the formula to all the codes now I got all the codes over here now select all the cells get into the fonts and here select the barcode font and there you go you have the barcode for all the numbers you have in your column A so that’s how you create barcode in Excel Excel flashfill is a feature of Microsoft Excel where Excel can sense a pattern in a cell and apply the same logic to extract the similar resultant pattern out of the remaining cells in a table might be a little confusing right so let us simplify it so let us imagine that you have a text in one of the cells in the Excel sheet and you want it to trim a part of that text for example let us imagine that there is an assembly an assembly where the car parts are assembled together to finish a car so each and every part has a code the serial number the product code and the code Etc right and you wanted to separate all those three if you wanted to do that you might want to use the trim function which might be a little tedious to apply to all the cells but what if if I say there is a simple key format using which you can fill all the columns in the Excel sheet within a fraction of second sounds interesting right so that’s exactly the flashfill function in Excel does now before we get started let me tell you guys that we have daily updates on multiple Technologies if you’re a tech geek in a continuous hunt for latest technological Trends then consider getting subscribed to our YouTube channel and don’t forget to hit that Bell icon to never miss an update from Simply now without further Ado let’s get started with the Flash Fill in Excel for that we might want to start the Excel to get in the Practical mode now we are on the Microsoft Excel and remember the example we discussed so this is a similar example now the first column in the sheet is the product detail so this product detail consists of the product code that is 112 next we have the product serial number that is 1 025 637 and lastly the assembly code that is as snx now we want to separate this product detail into three columns where we have the First Column to be storing the product code second one to be storing the product serial number and the last one to be storing the assembly code so now if you want to do that using trim function you might want to apply the trim function here and trim out the left part and again for the product serial number you might want to use certain function again which might go into a little complicated pattern So to avoid the complications and and decrease the time spent we might want to use the Flash Fill in Excel now let us help Excel about the pattern that we might want to generate so the pattern is one12 and the next one is 322 so now you can see that the text was highlighted that was the intimation from the Excel that it understood the pattern now going into the NEX Cel and holding the control key and pressing the e button will help us to fill the remaining cells in the column B so that’s how it is done and similarly let’s try out for the serial number for that the serial number is one2 5 637 now in the next cell we have 13 6582 now we can go to the third cell and press CR e for Windows and if you’re using a Mac operating system then you have to press command e and all the serial numbers will be filled so this will reduce the time in separating the pattern and filling the columns now let’s try the last column that is as SX which happens to the assembly code next we have FG and V now the shortcut method that is contr e and there you go now you might be wondering will it work only for numbers etc etc or the text which is properly aligned or separated using a hyphen no it can also be used for some random text which is like this so let us imagine that we have a text which is of length like 20 or 30 and you wanted to print only a part of it so here I’m going to print the alphabets from position fourth to position 8th so I’ve already tried the first one that is taking out the alphabets from fourth location to the eighth location or the pattern of the text that is i f e from Jennifer Lopez and in the next cell we have the numbers starting from the location 4 to 8 so that is 7767 and similarly we have the other cells consisting of the names of few cars that is Alpha Romeo Bugatti Von and next we have Superman Returns text so now let’s press the shortcut key that is contr e and make sure that all the remaining cells are filled so that’s how the Flash Fill works so the Flash Fill in Excel will reduce the time consumed to fill all the columns by separating the text in your main cell that is the column A which we have here in this example and it also reduces the complexity of using a trim function in Excel as hyperlinks in Excel now hyperlink is something which is clickable and when you click on it it will redirect you to a different web page to a different worksheet or a different location in your local system so there are a variety of options and we will explore all of them using Excel now creating a hyperlink is really simple we have multiple ways now let’s write some text on the Excel worksheet I’ll write Simply longan Now creating a hyperlink is really simple you have multiple ways the first simplest shortcut is clicking on the cell where you want to place the hyperlink and holding control key and pressing K now this will enable the hyperlink menu box or the dialog box this is the first way and let me close it now again we have selected the cell there is another option where you have to click on the insert menu on the toolbar and navigate to Links group click on the link option and you will again have the dialog box of the hyperlink and Excel and the Third Way is right click on the cell and you can see the link option in the last and this will enable the hyperlink option in Excel now let’s create a hyperlink the simplest way again control K now we have a variety of options that is existing file or web page place in this document create a new document and an email address so we will address all of these one at each time so first one existing file or a web page now you can select an existing PDF file or any file in your local system and hyperlink or create a hyperlink to the text or you can select a website link and place it over here and create it as a hyperlink Let’s uh explore both the options first let’s try to explore the web page option so since I’ve have written as simply learn let’s use the website link of Simply learn and create a hyperlink let me open Google new tab and let me type simply learn.com let me copy the hyperlink and now let’s get back to the Excel workbook now here let’s place the hyperlink now you can press okay and there you go the hyperlink has been successfully created now this link is clickable when you click on it it will redirect you to a new simply learns home web page right so this is how you create a hyperlink to a website now we shall explore another option that is using an existing file right let’s create a new text box here let’s name it as existing file and okay existing file should be good again the same you can either click uh you know right click and select hyperlink or control ke now let’s select existing file or web page option now here you can see navigate key press on that and let’s navigate our files let’s select all files you here remember to select all files okay by default it shows as office files now let’s explore our file option so let’s select uh fundamentals of computer programming option which is a PDF file press okay and there you go we have created a hyperlink so when I click on this you can directly navigate to an existing PDF file in your local system and it will be open as a web page click on it and you can see the PDF file now close now another thing when you navigate on to this you’ll see the lengthier HTTP uh address and same happens here right so if you’re a little uncomfortable watching this lengthier link you can also edit it you can choose the option of edit hyperlink and see the screen dip option click on it and here you can change the text display option and write it as click here to know more press okay again press okay and now if you navigate onto it instead of seeing a lenia hyper link what you can see is click here to know more okay I think I made a spelling mistake so I think I forgot an e somewhere over here yes click here to know more and press okay right so here you can see click here to know more let’s try this again over here so edit edit hyperlink screen tap option and instead of the lender text box or hyperlink you can write as click to open notes or PDF click okay and click another okay and you are done so instead of seeing that lengthier hyperlink you can see click here to open PDF right so so far we have explored two options that is existing file and existing web page right now let’s try to explore another option again uh hyper link right click hyperlink and here you have seen places in document all right so which is like uh okay let me show this to you instead of telling you so here what it is doing is you have the cell A5 and you want it to refer to A1 or any other cell in the same worksheet so you can do that right so now for your reference let me cancel it and let me create a new sheet and here navigate me to here right let me write as navigate me to here okay so now what we will do is so here we will create that navigate link from using the hyp link option place this in this document you can see the sheet to option and the cell reference is A1 where we have written navigate me to here so when I click this A5 cell okay let me also rename it write it as navigate cell navigating cell right now create that control K let me select sheet two and the S address is A1 press okay now what did uh what What’s the change right when you click on this you will be redirected to sheet to cell A1 navigate me to here right so this can be like a shortcut when you have multiple sheets like 100 sheets then you can uh you know have a a different column over here and uh add the navigation address to all those cell links you can just click on that cell and go into the different sheet and to the specific cell which you have given the address in our case I have given sheet to address cell A1 right so this is another trick now let’s get back to the hyperlink thing again uh control K and another option is create new document so you might have been through you know option of uh having what’s say some cheat sheets or uh clip on clip boats right so uh let us imagine that you’re working through uh a huge worksheet and uh you wanted to you know uh have a a track of everything and keep notes updating right so it might be a little complicated to understand so let me show it practically so let me write it down as clipboard or sticky note right okay I think this has a hyper letter us okay this can be used no worries back to uh hyperlink option again create new document now here you can create a variety of uh documents right so it can be a text document it can be an Excel doc document it can be a PDF document right for now we’ll create a text document okay uh we’ll name it as uh tracker. txt so this should create as a text document and the location can be anything so you can you know uh take it as a desktop documents download so let me take documents here and select okay so I’ll be having the tracker. txt located in the documents folder and uh you can also choose to edit the document later or edit the document now let us select edit the document now press okay and there you go so you have your text tracker here so let us imagine that we have worked on the Excel sheet and we wanted to write on us a sticky note uh so that we can remember what we have done on the Excel sheet so update one and uh update two now you can close it so ask for a save option save it and done and let us imagine that you did something else and you also updated something else on the sticky note I mean Excel sheet and you wanted to update that on your sticky note so click on it and you’ll see a minor notice over here from Microsoft Excel based on potential security concerns you can select yes and you’ll be having access to your text document again and write down your update again here update three what you have written right the latest update what you have and you can save it and close so that’s how you keep track of your sticky note and you’ll have all your updates right now getting back to the hyperlinks again uh the last one is email address so you can also try to add uh okay let’s imagine that you’re working with a colleague and every time you make an update in your Excel sheet or anything and you wanted to send him out an email right so it would be a little timec consuming that you open your email and uh compose something and send it over here but what if you had a one single click and access to the email compose option right that can be done so let us write down email c b me with the spelling so again the control K and hyperlink select email option and you can write down your email which can be anything so you can write down your friend’s email address so I’m just writing down some random email here friend gmail.com and can write the subject about update and press okay this will navigate you to the Gmail option or the mailing address options click on it and you’ll have a variety of mailing options Outlook Office 365 Yahoo iCloud Google Etc so you can select any one of those since we have written as gmail.com we’ll navigate to Gmail and all you have to do is login you know all the formalities and you’re good to go with emailing so let’s skip this for now so you you know how to do the rest parts so so far we have uh you know covered all the hyperlink options in Excel that is existing file or a web page place this in the current executing worksheet or document creating a new document for your update thing and uh also including the email right so so far we have covered everything around hyperlink and Excel and if you have anything to know more about or if you feel that we have missed out anything important regarding hyperlink in Excel you can feel free to let us know round by formula in Excel now for this particular tutorial we will be using the student data set now let’s get back to the Practical mode and start our Microsoft Excel now we are in the Microsoft Excel and as you can see on my sheet okay let me expand this so as you can see on my screen we have the data set belonging to students so we have 10 students and every one of them has name role number class blood group and marks Etc and at the end we have the percentage and as you can see we have the percentage in terms of float values and we have a lot of decimal values like eight and above so what if you wanted to just have one or two digits after the actual percentage like 80.6 or 8.66 right it would be a little bit more or good to read right so that’s what a roundoff actually means so now let’s work on having rounded off values for the percentage now let’s create a new column let’s name it as um round of percentage great so to actually perform the roundoff uh formula we do have a pre method for that for that you just have to type in equals to round and there you go we have round up and round down so both of them perform the same operation let’s select a round up and this is the m column and the row number is three so we’ll have to press M3 and the number of values after the decimal point for now let’s select um one and there you go press enter and the value should be rounded off now 80. 66 is being rounded off to 80.7 now when you drag the cell to the all the rules then you can see all the values will be rounded off here you can see 75.1 16 has been rounded off to 75.2 and 82.3 3 has been rounded off to 82.4 so we end up with having rounded off so first we will understand what exactly is standard deviation so standard deviation is a calculated square of variance now what is variance okay nothing to worry let us also understand what is variance now the next part variance so what is variance variance is a measure of variability it is calculated by taking the average of squared deviation from the mean now what are deviations so slowly you can understand that we’re getting into the topic of statistics and graphs so uh before getting deep into it let’s understand what is deviation so the deviation is a measure that is used to find the difference between the observed value and the expected value of a variable in simple terms deviation is the distance from the center point so for example when you are going through a graph so you will be expecting some value right and if you get some difference that is above or below the expected point that is called the devision what is the difference between the expected point and the obtained point so that is the deviation now next is the observed value so the observed value is the value that you get in real time unlike the predicted value so now you might be thinking to calculate standard deviation we might need a few more parameters and you’re exactly right so to calculate standard deviation in Excel you need variance and then you need deviation and then you need mean right so all these parameters are supposed to be calculated first and then you will be having the final formula to calculate standard TV now that we understood the theoretical and formula based explanation about standard deviation it’s time we calculate the standard deviation using one of the most popular business intelligence tool that is Microsoft Excel so let’s get back to the Practical mode so now we are on the Practical mode and we have some sample data on my screen right now let me expand it a little bit yeah there you go so as discussed before to calculate standard deviation we might require the mean variance deviation and deviation squared and then we can calculate the standard deviation so let’s go step by step so the first one is calculating sum right right so to calculate average we might need some of all the scores so we have the index values that is 1 to 10 and scores of each index value that is 1 to 10 right now let’s calculate the sum of all values here press tab to select the function select and drag these cells press enter and then you have the scores now you want number of indexes so clearly you can see we have 10 but still let’s count them C N tab space to select and and select the number of cells press enter and you have count now what’s the mean so mean is simply the average of all the numbers right so you can either divide the sum by count or you can simply use the average function so let’s try to use the average function so that we also have an idea of how the average function Works press tab to selected and select all the cells press enter now you have the average or mean value now we have the mean value and now after finding the mean value we are supposed to find out the deviation so remember the deviation that is the difference between the obtained value and the predicted value right so the obtained value is this one which is here in C2 and the predicted value is this one that is 82.6 that’s our mean right according to formula we supposed to eliminate the value of obtained minus the mean right so select uh the cell equals to C2 minus the obtained value that is mean now press F4 to freeze the value and now press enter so that you can freeze the value and drag the formula across all the sets now you have the deviation right now the next step is to find the sum of all the deviation numbers so either you can apply the sum formula to all these numbers or you can just drag this cell over here and the formula will be automatically copied now similarly the count function to count all the values that us drag it and you’ll have it over here right now the next step is to find out the square deviations or the square of the deviations which is really simple all you have to do is equals to select the cell and uh see that small hat kind of logo on the number six use that now into two press enter you have the square similarly drag the same formula across all the cells and you’ll have your squared numbers similarly drag this summation function over here so you’ll have the sum of this and the count number as well now comes the final formula now we have the mean value that is over here let’s color it and we have the variation Square now that is which is this one let’s color it in a different shape and we have deviation as well let’s color it in a different shap now we need to find the variance and after we find the variance the last step is to count the standard deviation or calculate the standard deviation so what is the formula to calculate the variance so for variance you need the sum of deviation Square divided by total number of values minus 1 press enter and that’s your variance now the standard deviation so the standard deviation is really simple you need to calculate the variance to the power of5 and that’s your standard deviation so that’s how you calculate standard deviation in Excel I hope all the formulas and the explanation was clear so you can see that we have got started with Microsoft Excel and on the left hand side we have a simple table with all the teams in an IT industry starting from it admin testing development to client consultant support we have got everything in this particular table now our main idea is to find out the index of these particular elements for example if you wanted to find the index of the element marketing then how could you do it so here let’s try to use the match function in Excel for that we want look up value and position so on the right hand side you can see that I’ve created two separate columns as lookup value and positions so in the lookup value column we will be inserting the value of which you want to find the index now for example let us consider that we want find the index value of Team marketing so I’ll be writing Marketing in this particular lookup value now how to find the index value of the element marketing for that we will be using the match function in Excel now after writing the match function you can see that it is asking for three different parameters so the first one is the lookup value second is the lookup array and lastly we have match type so lookup value is the value of which you are trying to find the index that is marketing this particular cell so we have selected the lookup value which is the first one next we have to select the lookup array that is from which set of elements you want to find the index so the set of elements are this it admin testing the column which we have created the the a column now we have selected the lookup array and lastly the match type is it you’re looking for less than or is it you’re looking for exact match or is it greater than match so for now I want exact match so I will be selecting the exact match that is zero now there you go you have the exact index of the element that is five now we have selected all the 10 teams starting from it1 admin 2 testing three development for marketing five so that’s how you find the index of a selected element in Excel now we just have like 10 elements so we have got the uh writing of the elements or you know uh choosing the elements in a easy way but what if if you had like 100 elements right could be a little different okay so let me put it simple now you wanted to find the the index value of Finance so right now we have marketing and you wanted to type in finance and you missed out a letter A okay now you’re not finding the right index to avoid such problems you can always take the help of data validation in Excel that is using the list in data validation in Excel you can see a small drop- down icon here right you can choose that and here you can see all the tees that we have created in the left column right so here I can scroll down and choose the finance and there you go I have the index value okay again we need to copy paste the same match function in Excel over here equals to match look up value look up array and exact match so there you go that’s how you make use of data validation in Excel and using the list you can comprehend all the list elements in this particular column into the list of data validation and you can choose the team whichever you want to find the index for so this is how you can make use of match function in Excel or index match function in Excel to sort data in Excel so on my spread sheeet you can see some employee data and on the column f and g you can see employee date of birth and employee date of joining for example you wanted to sort the employee data in the form of earliest joining employee and the latest joining employee right that is who is the employee who joined first to this company and who is the employee who joined last to this company so for that you just need to select the column of date so let’s select both the columns and go to home and check or verify if the data type is date or not so generally the data type will be set to General by default by Excel so we need to make sure that we have the proper data type which is State now that we have the proper data type select the entire column and go to the data option and here select the sort option and in the sort option select the expand selection click on sort and here sort by employee data joining and oldest to newest that’s what we wanted the oldest employee or the you can also set it to the newest employee to oldest according to your requirement but we’ll go to oldest to newest according to the case study right now just click on okay and that should be done so Emily who’s a manager is the oldest employee who joined the company and Chris who is A traine in it support is the latest employee who joined the company now that’s how you sort date in now on my screen you can see the sales of four quarters from different regions that is east west south north and Central now you might want to calculate the sum of sales happened from all the regions in q1 or you might want to calculate the sum of sales happened in one region of all the four quarters that is either you want to calculate the sum of all the regions from quarter 1 or you might want to calculate the sum of all the quarters in one region right so to do this you have some simple functions in Excel so you might say me you have some function Etc right

    for that you might want to go into the you know equals to mode and fetch some option and subtotal or some you have to select the function from here but what if I tell you that there is one simple easy shortcut where you just have to make one single click and you’ll have all the Su right so that seems interesting so on the top in the home bar select home and in on the top right corner you have the editing group in editing group you have aggregation function that is summation so you have different aggregations here some average count numbers maximum minimum more functions here you might want to select the sum and you can get all the sumission numbers here so I’ll select one cell so the second query where we wanted to calculate the sum of all the sales happened in each region with all the quarters included that is this one C2 to F2 right so let us select G2 and select the auto sum option and here it will automatically select the range for you that is C2 to FS2 if you want to change it can also you know change it like minimize the number if you just wanted to calculate the sum of three quarters you can do that so now we want four quarters so select enter and there you go you have all the sales of all the four quarters of each east region now if you simply drag it you will have the sum of all the four quarters of all different regions now let us calculate the summation of q1 of all the regions that is this one Select Auto Sun and it will automatically select all the cells for you that is from C2 to C6 right press enter and you’ll have the summation similarly you can you know drag it and you have the numbers here of all the Q2 sales of all the regions Q3 sales of all region and Q4 sales of all region now let us try to change the color that you can have a reference that is it’s different it’s it’s a total sum or right you just have a reference to identify it now similarly to this one as well so that’s how you implement Auto sum in Excel now we are on the Excel spreadsheet so you can see that I have five different sheets of the same data so why do we have that so I’ll be explaining you how to sort data for multiple parameters so firstly we will try to sort data in the form of numbers right so let us imagine that this data is being shared with you by your manager and he wants you to sort this data based on the salary numbers so so he wants you to arrange the data in the form of ascending order or descending order maybe which will help him find the employee with the highest salary right so how could you do that so it’s really simple you could sort the entire data just within a few clicks so how would you do that so since you’re focusing on salary select any sell from the salary column only right select this particular column and select any sell in the salary column get into the s filter options and here select this sort of arrangement you need that is largest to smallest or the smallest to largest since the problem statement was to find the highest salary click on largest to smallest and there you go so you have Tony and the designation is senior and he works in it support and he draws the highest salary of 80,000 right and there’s also another one which is Peppa and she works for the analytics team and now the next sheet let’s get into the sheet two and this time let us imagine that you wanted to you know uh arrange or sort the data based on the employe date of joining or date of birth right again if you wanted to arrange the data in which you find the youngest employee or the oldest employee of the organization you can do that click on any cell in the date of birth column and go back to the sort and filter and here you will be seeing the options sort oldest to the newest or the sort the data from newest to the oldest so since we want the oldest employee select oldest to newest and there you go so the oldest employee is Alfred who is an associate designation employee and he works in the admin department and he’s drawing the salary of 25,000 so that’s how you can sort the data when you are trying to sort the data based on date wise so first we try to sort the data on the number wise so you can apply the same for your employee ID salary and if there is any possibility you can do that as well and if you want to apply the same sorting based on dates you can do that as well using the Sorting function in the same way now let’s get back onto the third sheet and here now uh single column wise is perfectly fine there is no issues with that now you want it to you know uh let us imagine that you have two columns okay let’s eliminate a few columns here maybe we can eliminate the employee ID and uh okay let’s keep a few columns so we’ll just keep employee name and employee salary okay so here we are trying to demonstrate if you are uh trying to sort the data using one cell in the column B will it maintain its relationship with the cell a okay uh that’s our query right so let’s see I’ll go a little detail so let us imagine that you wanted to sort this data based on highest salary okay so when you do that will it also impact the cells in the column A so it will basically impact but just uh it’s a query right any of you viewers might have that query will I uh if in case if I try to sort the data based on this will it affect the cells on the rest of the data set okay so in basically it will impact so I’ll try to you know sort the data based on ascending order so this is the smallest to largest and you can see uh the Tony and pepper so since I selected the cell in the column B it also impacted the change on the cells of column A so that’s how the Sorting works now we’ll get to a little Advanced um sorting techniques so here we try to I mean already we try to sort the data based on numbers based on alphabets based on U you know uh date date of birth right so I think we forgot the uh ascending or you know sorting based on alphabets let’s do that as well select any um cell in the column A and go to S filter try selecting a to z or okay let’s try to Z to A okay so here you can see the U names have been sorted based on the descending order from Zed to a right so so far we tried on alphabets so far we tried on numbers date of birth and a couple of more things so let’s dig a little deeper and try to sort based on colors right so for that you don’t uh you know find that uh extra setting here in the regular sortant filter so you might want to get into the data option and here you’ll find another sort function which is a little advaned so that’s how you do it so since we have colored in the cell or column of salary select that salary and we’re going with cell value so in the cell values section select the cell color and and here on the cell color options you’ll have green blue and yellow that’s what we have used in our um cells so let’s keep Green in the first position which is on top and press okay now you’ll see green has been sorted to the first position now yellow and blue are left out so why is that so because you didn’t provide it right so you can do what that as well I mean you can always add uh the setting to it right select any cell go to sort and and add a level and then again select the salary and sell values to cell colors and now the cell color in the second position would be blue and that will be on bottom and I can add another level the same salary the cell values will be cell color and uh cell yellow will be somewhere in between or bottom okay so based on hierarch key uh it will be selecting press okay and there you go so you have sell yellow and blue at the bottom and green at the top position as for your suggestions now we are on the last sheet so here in this last sheet we are going to you know um dig a little deeper into the advanced sorting methods so previously we tried to apply the advanc Sorting methods for colors now we’ll try to apply the advanced sorting methods based on the data set we have here I mean the data we have in our data set right so select the sort option and let’s add a few details right so firstly let’s sort the column employee ID okay so let’s have the smallest to largest or yeah now that would be good now let’s add another level where we will sort okay uh I think we will sort based on employee name that would be a little better so let’s try to uh sort it on employee name here and here will be employee salary okay now the sell values and this would be from a to zed and the salaries will be from smallest to largest so basically what we are trying to do is we are trying to sort the employee list based on employee names from A to Z so the employees name starting from a to zed will be ordered first and another subset of the Sorting will be done based on highest salaries that is smallest to highest so let us imagine that we have three employees with Nam starting from a and their salaries are 10,000 20,000 and 30,000 so how would it sort so it will sort basically uh in the alphabetical order first so first three names will be sorted and then it will also have another condition to sort the data we keeping the lowest salary on top and highest salary in the last right so uh this is how the algorithm works for sorting now let’s press okay and see the output so there you go the names are Alfred and you have the highest salary here smallest to highest so this is 25,000 next we have Banner Barnes and Ben so here you might be seeing why we have Ben in the third position despite he has a low salary and name starts with B right so here it is trying to work based on the alphabets in the order right so B A and B E it’s not just comparing the first letter it’s also comparing the following letters as well right so n comes first R comes later so B is pushed to the second position a comes first and E comes later so B A letters r pushed to the first position and Ben is pushed to the third position so that’s how it works right so accordingly the set for BA is created here 3540 next Ben 12 and next you have Bobby Brian Brock Chris clar right so that’s how the data has been organized and sorted based on multiple levels of parameters you provide for sorting so that’s how you use sorting in Excel slicers and filters in Excel so slicers in Excel are software filters used along with Excel tables or Excel pivot tables over a large amount of data not just filtering out data but slices also help you with an easy understanding of the information being extracted and displayed on the screen now Microsoft Excel slices are compatible with Windows and Mac and tou operating systems now let us understand how to implement slicers in Excel so for that we might want to get back to the Practical mode that is starting our Microsoft Excel now we are on our Excel sheet and as you can see on my screen there is some data available on the Excel spreadsheet and this table is not just any table this table from Microsoft Excel is been converted into an actual table as you know by default Excel considers all the data which has been inserted into the spreadsheet as a database and to implement this sense in Excel we might want to create or convert the format of database into a normal table for that you can select all your data and just press contrl T and that will allow you an option called convert the data into table now since this table is is already converted as a table we can directly start implementing or inserting slices into the spreadsheet now for that select all your data then going to the insert option and in the insert ribbon you can see the filters group and in the filters group select the option called slicer now you will be provided with multiple options since we have the columns employee ID employee name Zone designation Department salary employees date of birth and employees date of joining so we can choose any of the options available according to our columns and create the slices now let me select the first one that is Zone next designation department and let’s also select employee name now press okay and there you go we have our slices now let us try to rearrange these slices all you can do is just select and drag them up and done now let us imagine that we want to see or take a look at the employees who are working in the Easter Zone then we have all the information displayed on the screen who are working in the East Zone they are Jack Tony bner Fred Etc and their Department designation etc etc all on one single screen so this really helps you while you’re presenting your data in a presentation now for a change let’s try to select the department now if your client wants to select the data from only analytics Department then you might want to select on the analytics key on the department SL ISO and there you go you have an employee called Luke hops who works in the East Zone and his department is analytics and his designation is contract based and the salary he is getting is 65,000 now similarly uh you can see that we have only selected analytics and your client also wants data from itop now you can see that analytics is gone but what if you want both analytics and it support then you have an option called select multiple then you can select analytics and it support and if you want HR you can also do that and in the zone if you want West along with East then you can do the same east and west and you have all the employee details and in the designation let’s try to select U senior traine okay select multiple senior traine and manager so that’s how you do it so that’s how it’s done and that’s how you implement slicers in Excel to simplify the filtering options using slicers during a presentation it is really simple to add filters in Excel so we are on a practical mode right now so on my screen you can see an Excel spreadsheet of sales data so here you have various columns regions category state subcategory sales quantity and much more right so now let us imagine that we want to add filters on the region part so we have south west east Etc right so we want to you know let us imagine that you want to extract the data of only vest region and you wanted a filter so doing it manually could take a lot of time so using filters will be helpful now select one cell on the headers part and select the filter option in the data toolbar you can see the data option on the toolbar right so click on the data option and navigate to sort and filter uh group and in that select the option of filter now you can see the filters are been added to all the column headers region category state subcategory sales quantity right now we wanted the West category so when you see or when you click on the drop down icon you can see all the options are selected right the central east west south every everything right but now we just want West one so click on West and select okay there you go so you have everything from the west region now let’s imagine that you wanted to look into uh office supplies only or technology only right you can do the same with the technology ology and press okay and now you’ll have all the technology related sales in the west region right so that’s how you use filters in Excel or that’s how you add filters in Excel goalseek in Excel so for this tutorial we’ll be considering the students database now let’s get back to the Excel so now we are on the Excel sheet and you can see that in this particular Excel sheet we have some student details we have the serial numbers names role numbers class blood group and some subjects like math math science computer statistics social GK and the final marks obtained percentage and total marks right so let us imagine that uh a company has come for an interview so all these students are going to attend an interview and uh the minimum percentage to attend the interview will be like 75% so you can see that we have 75 here and all the students have got above 75 except for the one student which is Mike right so for Mike to attend the interview he needs 75 percentage or above so uh he can make some modifications in one of these subjects right so there is always an exam after the exam called as Improvement exam so if you can like to score some extra marks in the subjects which you have scored less so that you can make up to the percentage what you’re expecting you can do that so here in this particular sheet uh Mike has six subjects right and in subjects you can see he has scored 76 91 45 71 94 and 62 so out of these he have scored Less in one subject compared to all the subjects one subject is really low that is computers so what if he can give an improvement exam and increase his percentage to 75 right so that can be done so we can do some root Force methods like we can change the marks like 55 and then check this particular one so it’s getting 74 right so using the brute force method will be a little bit lengthy so you can do this because it is just a small uh table which has least number of data and you just have one single cell to modify so that can be done in small tables but what if you had some table with hundreds of rows or thousands of rows that’ll be time consuming right so for that we have some inbuilt functionality in Excel that is called goal seek in Excel so for now let’s eliminate the marks here so we have eliminated the marks and the upgraded percentage is 65 now we need the target as 75 okay so let’s uh write it as Mike’s Target right we need mic so mik new Target as 75 okay now to make sure that we need a Mark here which can get mic to 75 percentage we will use the data operation so here on the toolbar which has file home insert Etc we have one other option called as data so when you click on data from toolbar you’ll have the ribbon and in this ribbon we have various options so when you come into forecast group we have an option called what if analysis when you click on wtif analysis you’ll have a set of options out of which we have goal seek now for goal seek we need to select a table cell that is this particular table cell that is M9 so we need the value 75 here right so what is the cell that you need to change here for that you need to select this one and select this particular cell here right and now you can select okay now you can see that Excel has automatically run all the permutations and combinations and has come up with a number so that the overall percentage of mic will be 75% right so the expected marks that Mike should be getting in his computer’s Improvement exam should be equal to 56 or greater than 56 to get the the final Target at 75 so that he can attend the company’s interview so this is how we use goal set or goal seek in Excel we are on our Excel spreadsheet and here you can see I’m trying to create an calendar for the employees in the company where you might have to you know write down the day and month and year or week of your birth date or maybe your joining date or maybe also your last working day right this might be useful for all the employees to you know uh avoid them from writing it down manually so how do I create it so it takes a few simple steps all it takes is some data validation methods and also there is an exclusive video specifically made on data validation in Excel link to which is in the description box below or you can search it on our official YouTube channel now you can see that uh the word calendar is on the first cell that is A1 but we have four cells below it that is A3 B3 C3 and D3 so you might want to add this to all the cells right like may be in the middle so for that you can do the option of mergin Center also there is a tutorial on mergin Center about all the options available in this particular thing so you can go through that also and uh yeah we have done the first step now all we need is day month year and week so in a calendar or in a month you’ll have days from 28 29 30 or 31 days right so let’s create a list or the drop down menu for that so that whenever you click it right so whenever you click on that particular cell you will have the list of all the days available right so let’s get started with that let’s create a new sheet so why are we creating a new sheet you won’t want that all the numbers in your calendar sheet right over here right so for that so we are creating a new sheet and all the data will be available here so here let’s write down the days first and here the second number also you can use the option of Flash Fill so that you can you know uh reduce the effort here so you can see the flash full option over here all you need to do is select the sales and drag them down until you get your required dates right so let’s keep it till 31 days there you go you have the days so these will be your days and here now the next option was month right yeah month and year now using the same flash F option you will you know add the months into your sheet so this will be Sunday so Excel has this smart flash F option where you can just drag the data and everything will be filled according to your requirements so Sunday to Saturday now the next one right that is your month so same as the before process we’ll create January now you can use the flash Bill option and all the way up to December now the last one was the year so 1990 and 1991 so you can select the cells and Flash Fill Up to 2022 maybe yeah 2022 should be good enough now this is The Flash option you have here and also we have an exclusive tutorial on flashfill all together you can go through it for a detailed explanation as well now coming back to the original sheet so this is where you create your drop- down list right so select the cell where you want to add the drop down list navigate to data and then here is your data validation menu so we have a variety of data validation options over here there is an exclusive tutorial on that you can go through it for now we’ll look into the list option only because we want to create a drop down list right so here it will ask for source click on the source here the source bar right now navigate to sheet number two where we have the days options select the range so this will be your range and select okay also you can add some message here right what you can do is uh in case if there is an error or something right you don’t want that to be happening so all you need to do is uh write down and select from dropdown let from drop down only so this could be a message select from dropdown only the title is Select options or just options in case if there’s an error you can write down your title as invalid data and you can write down a messag as please select from drop down only so um in case of someone tries to write as 32 or 33 to check the Integrity of your data validation then they will encounter an error and an error message and then they must be understanding what exactly they try to do and if it’s correct or wrong right so this is to make sure that whatever the data you enter is correct and valid so okay and here you can see select from the drown list only options with with option list as title and let’s try to enter 32 right and when you press enter you will see an error so the error messages invalid data please try to select from drop down only so you can go for a retry and select the drop down right so let’s eliminate them and now select something from the drop down so let’s take it as maybe 10 right so this is how you cleared the day one now let’s try to do with the month same data validation option settings will be list and Source will be this one think I made a mistake here so let’s cancel this and go back to sheet 2 and rename this as our week yeah now coming back to the month again it will be a list and the source will be in sheet 2 navigate to the sheet 2 and select your range and input message the same options select from dropdown only and error alert title will be invalid data this one will be please select from dropdown only it’s okay and that’s done now let’s continue with your as well let’s quickly do the same it will be a list Source will be in sheet two this is your list press okay and lastly the week same process it will be a list Source will be in your sheet two input message options tile invalidator strong down only right so your drop down list is ready now so the month will be anything So currently we have me and year can be anything anywhere so let’s take it as 997 and week can be you know the adjacent week so what was 10th of May 1997 let’s check our calendars was a Saturday so it was a Saturday and that’s how you create drop down list in Excel and if you want you can also add borders to your calendar right so that’s how it’s done so what is data validation in Excel so it is a feature by Microsoft Excel where it can restrict data entry into certain cells by using data validation and it will prompt the users to enter valid data in the cells based on the rules and restrictions provided by the creator of that particular sheet now this would be little confusing to understand so to understand it in a much better way let’s get back to the practical way where we will start entering some data validation rules to our Excel sheet and try to enter data into it now we are on the Microsoft Excel and this particular sheet is based on employee ID employee name employee Department employee salary fiscal year and work timings of the employees now we will be applying some data validation rules to each and every column and try to enter data based on the rules we have entered now for the first one which is the employee ID we’ll enter rules based on the employee ID number so to apply the data validation you need to go into the data tab on the toolbar and inside the ribbon you can see data tools group and in the data tools group you have the option of data validation now select the entire column go to the data tools group select data validation option and you will have this particular popup menu and here you can see different types of options so since we are entering employee ID it would be a whole number so let us select whole number option and now here you can see the option of between not equal to equal to not between greater than less than Etc so let us select between and let us assume that your employee ID has five numbers and uh it should be between 10,000 to 11,000 so let us provide 10,000 and 11,000 now select okay and now the new rule has been applied so now the minimum value of each and every employee ID should be equal to or more more than 10,000 and less than 11,000 and it should have five digits now let’s try to enter a wrong employee ID that is just one there you go you can see that Microsoft Excel is not allowing you to include a wrong number it says this value does not match the data validation restrictions defined for this cell so now you can press retry and try to enter some number which is in between 10,000 to 11,000 now it is 10,901 and this will take the entry now let is assume that because I’m the creator of this sheet I know what should be entered here but what if I give this sheet to you and you’re the person who’s trying to enter the number and it shows the error and you don’t know what was the error right to avoid this kind of confusion what you can do is you can do some uh ways where you can provide the message to the user so that can be done by okay let’s select the sheet again and go to the data validation and here you have the same settings now here you have another option saying input message so before input message let us go to error alert now here the error title would be data entered is not valid please enter this is the error message please enter EMP ID between 10,000 to 11,000 press okay now when you try to enter the wrong number it will show that particular error alert message see data entered is not valid please enter employee ID between 10,000 to 11,000 now this is fine but how long this will be you know what if there is an option where you just hover over to this cell and it automatically tells you to enter the value between 10,000 to 11,000 without having to face the error yeah even that can be done so you can select the entire column go to data validation and here this is the input message this is where you can do that so here you can write the message name valid data please enter data between 10,000 to 11,000 now when you hover over to any cell in this particular column it will automatically show you this message where you can avoid all the errors just select okay see when you select any cell in this particular column it will automatically tell you please enter data between 10,000 to 11,000 so here you don’t have to face any errors right now let us try to enter the valid data 109 02 and it will take it as the correct value and again 1 903 now in case if you try to provide the wrong number it will show show the error see so that’s how it is now we have finished the whole number part now let’s get into the employee name where we have to provide uh the second type that is text length now let us imagine that uh in your company you’re trying to provide ID cards to your employees and you know that ID card is really small and inside that ID card you need to include the employee photograph and your company name employee idid blood group phone number address everything and also you need to include the name of the employee so what if there is an employee with very lengthy name like 30 characters 40 characters yeah in India it is really possible that you might have a lengthy name so what you can do is you can provide the text length where the text is limited to like 15 or 20 uh um characters so you can include that name in the ID card so you can do that by allowing the data validation criteria with text length uh minimum can be anything so minimum can be one and maximum here you can provide 15 and press okay okay minimum will change it to at least uh two and yeah let it be one so minimum is one and maximum is 15 characters and press okay and uh let us also provide the input message enter character type character data please enter valid name and error alert invalid data please enter valid data okay please enter characters less than 50 press okay now let us try to enter a random name okay uh we’ll enter characters more than 15 here qu z uh yeah it’s like 1 2 3 4 5 11 okay okay now this is more than 15 okay I think it’s not more than 15 let’s try to enter a little bit more characters yeah now we have entered more than 15 characters and it is showing please enter characters less than 15 so now we can retry and try to enter some valid name um now we have characters less than 50 so it will take as the proper name now let us try to enter another name I’m just entering some random data so another name maybe great now the next type of data validation is also done now let us get into to employee Department employee department is something really superb so I’ll tell you about it it’s actually a list now let us finish the other ones first that is employee salary physical year and work timings after that we’ll learn about uh the list one now let’s come into employee salary so salary is something where you have to include decimal points so now let us go to uh data validation and inside the values we have already dealt with whole number uh text length yeah now we’ll uh deal with employee salary that is decimal now minimum is okay what is uh minimum can be 1. 0 and maximum can be 1 lakh or let us put that as 10 LHS press okay okay not not just them enter valid salary okay now let’s change it to minimum is 10,000 so the employee minimum salary will be 10,000 and input enter valid salary enter salary between 10,000 to 10 lakhs let us include a comma here too now error alert invalid salary please enter between 10,000 to 10 lakhs press okay now the message and the data validation conditions are apply to this column now let us try to enter some invalid data first it will show the error now we need to enter the valid data yeah now it is taking the valid data let us provide 20,000 and this has 35,000 now the next one is fiscally let us imagine that we wanted to you know work on the employees for current working year that is 2020 to 2021 and or you can 2021 to 2022 not more than 2021 and not less than 2021 for that you can provide the year option as well so for that you need to select the entire column go to data validation settings and inside here you have an option of date so uh let us provide the date option as between and start date is 01/ 01/ 2021 and end date is 01 sl01 SL 2022 and input message let us provide data as current fiscal year Current financial year enter date between 20121 to 2022 let is call copy this and go to error alert error message will be S invalid date select okay now the error message and uh the data input message and the data validation rule has been applied for this column so let’s try to enter some date here you can enter uh 02 of February 2021 it will take as the correct data now let’s try to enter some wrong data which is apart from 2022 02 02 20 22 it will not take this it will throw error so that’s how it works now let us try to enter uh 2021 and this is March it will take it now another data 04 this time and 2021 so that’s how your uh date data validation will work now coming into the next type of data validation that is time so we have time here let us imagine that you are providing some work timings to your employees that should be from 9:00 a.m. to 5:00 p.m. not more than that so you want to keep your work life balanced for all your employees so you just want them to work between 9 to 5 and that should be fine and anything apart from that time should be invalid so you can do that by selecting in uh time option between it should be in between and the start time will be 09 0 0 and this should be uh 17 that is 5:00 p.m. in the evening and input message please enter time between 9 a.m. to okay 500 p.m this should be the input message now error alert invalid time error message okay now okay work timings now this should be okay we can do this as two columns actually let us cancel let us copy this and paste it here you can do it as login login timing and this should be okay this should be a log out so this should be okay so this will be considered as login and log out times now let us provide the login time this should be the right time and uh let’s try to provide wrong timings 08 now it will not take it it will throw as error so it’s working fine let us provide 10 now let us provide the log out time it should be below 5:00 p.m. so let us provide five it will take it okay let’s try to provide something just 1 minute less than five that is 59 okay there is some problem in this let’s check it okay it is taking uh three that is including seconds okay let’s provide seconds as well 05 0 0 0 0 okay still there is some problem okay we have entered four we should we are supposed to enter 24hour timing right 14 okay uh it should be 17 I think this will work yeah now we are following 24-hour timing here so let’s provide another timing that is 16 30 0 0 this will be valid and 1530 yes this is also valid now we have finished almost all types of uh data validation things that is we have finished with any value whole number decimal date timing test length and the last one is list so this is where things get interesting now let’s cancel this now select the employee column now if you’re running a company then definitely you’ll not have only one Department in it you might have multiple departments right so let us consider that you have a software development company so in that the basic departments will be software development team and software testing team so what you can do is you can provide a list so using that list you can just uh H over to that cell select the drop- down menu and inside that you can select the option so we will try to work something like that if we have selected the entire column selection okay yeah let us select the entire column again go to data validation the selection contains some cells without data validation settings do you want to extend data validation to the sales yes now let us provide the list option and here Source you can provide as developer and test press okay okay now input message select one select one error alert invalid data select from drop down only press okay now when you select the cell you can see small drop- down icon right so when you press it you will have developer and tester option so you can select developer for first one and for second one you can select uh tester right okay this is good for uh one or two options maybe three or even maybe like five but what if you have more departments okay let’s go to this uh second sheet where we have the department data so here we have like 14 departments no 15 departments or 15 employe type names so the first one is CEO second is developer tester quality analyst system analyst Finance human resource so so many options are there so you cannot keep on typing all of them right so here it is 15 what if you have like 25 like 50 departments right it will be tough so for this you have another option so let me select all this data and copy copy this from this sheet to our employee sheet let us paste it somewhere here let’s expand this okay now we have our uh list over here now let us remove the data validation from this column clear all okay now everything is cleared let’s select okay now let us erase this data as well now let us apply the list data validation again from scratch go to list now here we can select the source select this particular cell and drag it until here and it has been loaded now go to input message data options select one from the menu or the drop down copy error alert select only from the drop down menu invalid entry yeah now let’s select okay okay fine now you have the icon here you have all the provided options you can scroll down and select any one out of these you can select knowledge transfer for the first one and second one would be system analyist and the last one would be human resources now in the list everything is fine okay this is showing you the menu options and everything so but what if you provide this sheet to your new joiny or new employee in your company and by mistake he messes up something like deleting this okay so we have deleted systems analyst and when you click on the options here you don’t have systems analyst in that place you have blank space right so there is a way where all your U you know data can be messed up so to avoid this what you can do is you can actually um save this data in a different sheet like I did here like Department data you can hide this sheet or you can protect this sheet with password or something so that nobody can mess this up so now let us get back to the employe data and select this column and uh clear everything press okay and let us also clear this and this as well in this one too now let us eliminate the row our sales the employee yeah everything okay now we have basically removed this data from here now let us try to apply the data validation once again now here let us select as list and uh here let us provide the option let us go to next sheet let us select all these options and press okay okay let’s provide input message data options or Department options yeah now let’s select okay now let’s hide this sheet now the sheet has been hid hiden and you still have the options developer or let us provide CEO then we have developer then we have test so that’s how you can protect your how to protect and lock cells in Excel now why do we need to lock or protect cells in Excel let us imagine that there is an Excel sheet with really confidential data and you need to pass that data to your subordinate or your colleague to make some minor edits now let us imagine that the edits are supposed to be done to only one or two columns and the rest of the columns should be left as they are now there is a huge possibility that sometimes unknowingly or unwillingly there might be some edits done by your colleague or your subordinate So to avoid such kind of unexpected mistakes you might want to protect and lock yourselves that you don’t want your subordinate or your colleague to edit so before we begin be kind enough to get subscribed to our YouTube channel and don’t forget to hit that Bell icon to stay updated on the latest it Trends and Tech content now without further Ado let’s get started with our onepoint agenda that is lock or protect cells in Excel now let’s get back into the Practical mode and start our Microsoft Excel now we are on the Microsoft Excel and this particular spreadsheet is related to employee details yeah now the screen is visible a bit better now let us imagine that this is our confidential data and you want to edit only two columns that is the designation column and phone number column now let us imagine that the company has finished one annual year and now there are some promotions happening in the company and let us also imagine that the phone numbers of the employees provided by the company have undergone some changes so now the minor edits that you want to make are related to the designation column and phone number column now you can see that all the columns that is the blood group new salary salary hike current salary and name of the employee serial number employee number everything is editable so in such kind of scenarios when you pass on this data there might be a possibility where your colleague or subordinate might end up making some mistakes right so you want to avoid that so you can do that by locking the celles now let us rename the sheet now the sheet has been renamed successfully so now you can log the cells by the following process so in this process you have two major steps first one is to log the cells and the next one is to protect the cells by a password now when you select all the cells in this particular sheet you can just click on this edge here and you have all the cells selected and now right click somewhere on the sheet and you can see an option called format sales so in this particular option you have some options provided which is number alignment font and you want to go into the protection part so you can see by default Excel keeps all the cells locked now we want the phone number and designation to be unlocked okay so let’s cancel it for now and get back to the sheet and select the columns D and I and now let’s right click and go into format Cs and in the protection option make sure that you uncheck the loged icon now select okay and now okay so you cannot include the merged cell fine we have a cell here which is merged okay let’s remove that okay now again select the entire column right click get into format cells and uncheck the lock option select okay now we have finished the first part that is locking all the cells which you don’t want to get edit and unlocking the cells where you want to make some edits now the second stage is protecting the sheet now right click on the sheet name and you can see an option called protect sheet click that and now here you can see select locked Cs and also select unlocked CS so let us provide a password here so let us use some simple password so that we don’t forget that so I’ll be selecting 1 2 3 and now select okay reenter the password to proceed now let us type 1 2 3 again now select okay and now the sheet is protected now we have successfully locked and protected all the cells in Excel now let us verify that the cells you wanted to protect are really protected and locked or not so we have unlocked phone number and designation and everything else is locked now let me try and edit the blood group so when I double click or try to edit this particular sheet then I’ll be receiving a warning from Excel it says the cell or chart you’re trying to change is on a protected sheet to make a change unprotect the sheet you might be requested to enter a password so it says that this particular cell is locked and you cannot edit it now let us try to edit the phone numbers which we kept as unlocked so you can see I can really edit the unlocked cell here now let me try to change this phone number let me enter some random phone number so you can see the cell is editable now let us go through the designation column and try to edit this particular column now since since I said that there’s a promotion happening in the company so the deputy CEO is now the current CEO of the company and software developer happens to become the senior software developer and the tester as well let us imagine that he will become the senior and so on okay so this proves that this particular sheet is editable not the entire sheet but only the sense that we kept in the unlock mode okay now this is how you lock and protect your sales in ex you can see that I have sales data on my worksheet so uh let us imagine that your manager asked you to find the average of sales happening in your company right so let’s navigate to the bottommost cell and you know write it down as average here okay average sales so here this is a cell where you want to keep your average right so here we’ll be writing a small formula that is average and tab to select it now you must select the array of data right so this is my array since average is an array function and now press enter so there you go you have the average sales data that happened in your company so to understand the page setup in msxl we will go through a demo inventory in Excel where we’ll be considering some restaurant data now without any delay let’s get started with the Practical demo now we are on the Excel sheet and this is the data I was talking about the restaurant data where we have the First Column that is Rank and next we have the restaurant name followed by that we have the sales happening through that restaurant and then we have the segment category that is what kind of service does that particular restaurant offers now this is the list what we made and uh what if we wanted to represent this data in the form of a printed paper right so I guess you can see all the columns and rows here so you can see that we have almost like 250 rows so there is no chance or there is no way to print all those 250 rows in one page of course it is practically possible but when you try to read the content it’ll be so tough to read Because all the rows will be clued together so closely so that you cannot read it so uh you need to print them Page by page so and you need to also take care of the data is aligned properly according to the page size everything and even you need to take care of the margins of your page right so that’s what page setup basically means so we had a detailed overview of our data and what actually we’re trying to make now let us see how page setup is possible in msxl so to go to the page setup menu or go to the page setup options we have three methods let’s start with the simplest method so the simplest method is by just clicking on file and go to print option and you’ll see all the page setup options right over here and you can also go to print menu by pressing a shortcut method that is contrl p so you’ll directly end up on the page set of options now let’s look at the second method so the second method is you can see the toolbar over here right which has file home insert draw page layout formulas data review View and help so the second method we can use page layout option from the toolbar and you can see when you click on the page option from the toolbar you have a new ribbon over here so this ribbon is all about page setup options so it will offer you margins it will offer you orientation size print area brakes background print tiles etc etc right so all these options or menus in this ribbon belong to page setup now let us look at the third method so the Third method is something similar you have view option on the toolbar so just select View and you can see we have few page setup options over here as well the normal page break page layout custom views and we have grid lines formula bar heading zoom in zoom out select to zoom free certain panes right so all these are also the page setup options so we have the three methods to enter the page setup options now we are familiar with those now let’s get started with the Practical mode on P setup for that let me use the simplest method that is holding the control key and pressing P which navigates me to the print option now we have various set of options here we have no scaling normal margins letter portrait orientation cated print active sheets and note one for Windows you can see this option is the printers option for example example if you’re connected to a printer which is the hardware printer then your Ms XL will show the printer’s name here so other than printer you can also make some other arrangements where you can just directly print your complete data in the form of PDF and then you can export your PDF to your recipent that’s also possible okay now let’s get started with the first option that is print active pages okay to understand this better we need to create a new sheet not a problem so you can see that we just have one sheet over here so let us kind of select all the items and create a new sheet and try to paste it over here or let’s try to paste it over here okay now let’s try to extend this yeah this should be fine now let’s rename this sheet sheet uh or let’s rename it as restaurant sheet two for our reference now let us get back to the print sheet option so I’ll use the shortest method that is holding the control key and press P now you can see that here we have six pages in total from the active sheets so the current active sheet is the top 250 sheet so this particular sheet is the current active sheet so if you print the entire data in this particular sheet then we will end up printing six pages now let us select the other option where you have to print the entire workbook right so now you can see that there’s a change in the number that is 12 pages so what is a workbook so basically workbook is the collection of entire sheets in your Excel homepage so in this particular Excel homeage which you have the first sheet that is top 250 and the second sheet that is the resturant sheet to right so the combination of these two sheets make up a workbook so when you select the entire workbook option you’ll have 12 pages so that’s how we use this setting where you select the active sheets the entire workbook and there is another option where you can only print a selected item so if you want to do this then let’s get back here now what if your customer or what if your client wanted the top 10 restaurants only so you can select the top 10 restaurants by doing this now you have the top 10 restaurants let’s go to print so you can see Excel has automatically selected the selection only print the selected content only so you can see we have only the top 10 restaurants or top 11 restaurants that we have selected in our sheet and it is ready to get printed so this is how we use the first option now let’s get back let’s not select that and let’s make the pages normal now get back to the print option again now you can see the next option that is cated right so in cated you have different different options that is 1 2 3 3 1 2 3 and 1 2 3 and the next one which is uncollated 1 1 1 222 3 33 so why are these numbers present here and what does colleted mean anyway so this is your question right now the answer for this question will be a little similar to your examination Hall okay let us imagine that you are in examination Hall and you are the invigilator and you have 10 students in your room so you wanted to provide question papers to all the 10 students okay so you have your question paper distributed into three sheets that is you have marked questions and your questions are fit into three papers right in this scenario we have 250 rows and all the data is split into six sheets that is sheet one or page one page two page three page four right now if we wanted to print 10 copies then the arrangement of papers will be page 1 page 2 page three page 4 page 5 Page 6 in continuous order right that is CED if you wanted 10 copies of different different ways where you wanted to print the first sheet first then you wanted to print the second sheet second and the third sheet third that is you’ll have 10 copies of first page 10 copies of second page 10 copies of third page and so on that particular approach is called uncollated approach and if you’re printing the pages in continuous order for 10 times then that particular approach is called as collated approach so this is what we had to learn about collated and uncollated now let’s get back to the third option now let’s press again print now we have our data here there’s something wrong select all the elements control p yeah now we have our data back now we have the third option that is the portrait orientation okay so when we click that we have two options that is portrait and landscape so you might be having a good idea like when you try to click a picture in your phone we have two modes as well that is portrait mode and the landscape mode so when you click on the portrait mode this is how your data will be looking now to understand the difference let’s click the landscape mode so when you click the landscape mode this is how your data representation will change you’ll have a wider page to print so this is why we use the page orientation for now let’s keep the page orientation as portrait orientation and we have now the fourth option so what is the fourth option we have various sets of options in here that is letter and A4 size there are also some more page options over here where you can select the page types you want to print so basically you’ll be having few more options that is A4 A5 A3 Etc but here we just have two that is A4 and letter so basically you’ll get those options and followed by that next we have margins so here you can see we don’t have any margins for our data okay so now we can add some margins by setting up this one as an option like last custom setting or you can also choose normal wide narrow right so your page will be changed according to your margin size right so this is how you can change the margins now let’s keep it as default let’s set the normal one now we have the next option that is scaling option so here you can see we are trying to fit the page so when you’re trying to print these sheets using no scaling then the actual size of the page will be printed now you can also modify that you can fit and print these sizes so you can remember the first option that I said of printing all the 250 rows in one single sheet this is how you can fit it so fit the sheet on one page where you can see all the 250 rows are fitting into one page but the data is not readable so but we don’t want this type of an approach right so let’s keep it as the default one which is no scaling so these are the variety of options in print now if you wanted to print the all the six pages into the PDF format and mail it to your client you can also do that I think I have explained it in the first place so you can see we can do it by just selecting this option where you can print all the six p just and into PDF format and then you can mail them to your client and you can also select the number of copies you want you can increase the copies to two three and any number of uh copies you want for now I’ll keep it as one copy now let’s print this now let us select the location rename it and publish now let’s get back to documents and see if the page is printed or not here you go here we have the resturant data in the form of PDF so we have our entire six pages of data in the PDF format now we are back on the homepage now we had some limited functions in the print function over here we had try to export the data so you can also export the data or try to print it from the print option over here you can just select that as well anything F PDF yes the pages are printed now there you go we have the restaurant 2 PDF data so that’s how you can print all your data you can either choose the export option or you can either choose the print option now when you see into the print option we have limited number of page setup options over here now let’s check out what we missed so we are in the page layout here so we had gone through the margins we had set all the margins and then we have also gone through the portrait and landscape now you can see this size when we were in the print menu option we had only A4 and letter but here you can see there are a wide variety of options that is other than letter we have tabloid legal statement executive A3 right we have A5 and many more other options and if you want some more paper options you you can go to more and you can always select a few more options that’s how you do it and yeah we didn’t had the break option so remember when we had been through the print option we did not have the break option there let’s select this to normal A4 so you cannot see the break option over here right so here we have the break option so what does break option does right you can actually break the page uh remember we spoke about the top 10 things or the top 10 rows you can always do that so you can just keep the top 10 restaurants in your first page and then break the remaining and to set page breaks you can select the cell now I need the top 10 so I’ll select this particular cell which is in the 11th position and now I’ll try to implement the page break insert a page break and now you can see on my sheet there is a thin line which is separating the two pages that’s how you can see the indication of a page break implemented onto your page now how do you see it you can see it by SEL in control P where you can see the first top 10 restaurants being printed on your sheet so there you go we have the first top 10 restaurants printed on our sheets so that’s how you use the page break now let’s try to eliminate all the page breaks so you can select braks and reset all page breaks so the page break has been eliminated now you can always choose a background for your data as well you just uh if you want you can add a picture to your background I think that’s not available for now okay you can work offline as well let us select this one so you can see that we have added a background to our data we can select all the text files and then we can change the color so that the text becomes a little visible yeah that’s how you do it now if you don’t want the image to be added on your file you can also remove that delete background and there you go all the images are gone now the next important part that is the print titles now why do we need print titles for that let’s try to print this so now you can see the first page in the first page we have the title rank restaurant sales segment category but if you go to the second page you don’t have that title right you don’t have the title rank name of the restaurant sales and their service category so if we want that then we need to add titles now for that let’s select the print titles option now let’s select the row to repeat on top now for that we need the first row select the first row and this particular row will be repeating into all the pages now let’s try to print it so yeah now you can see the titles are being added to all the sheets you can see the page two page three page four page five and page six so you can see the titles added to all the sheets now we are on the spreadsheet so here what exactly are page breaks so page breaks are something which help you during your trying to print your data or present your data in the form of a web page right in those times you cannot fit your entire Excel data in one spreadsheet or one paper or one web page right so in those scenarios Excel will try to elimin a few parts of your spreadsheet or it will try to split the spreadsheet so how exactly it looks like so go to the view option and click on The View and click on page break preview here you can see the dotted lines right so these are the things which separate your data in form of pages so that it can be printable on one single page so some of the times the user or the xuser manually inserts a page break right so how do you insert it just go to the page layout and here you can see the page breaks insert a page break and you have successfully inserted a page break again go to the view go to page break View and here you can see the solid line so these solid ones are the ones which are inserted by the users or anyone who had the access to the data now how do we eliminate them right so we can eliminate them and we can try to you know get the shet back to the normal way so all you have to do is click on the the cell which has the page break which is right under the page break go to the insert page break yeah page layout menu go to the brakes option click on remove page break and there you go you have successfully eliminated the page break and page break preview to make sure that the page break has been successfully removed so you cannot see that anymore so that’s how you try to insert or eliminate a page break in Excel so conditional format is a method to visualize your worksheet we already have the charts in Excel to visualize our Excel sheets in graphical form but what if you had to see or visualize the data in your worksheet as it is so that is where the conditional formatting comes handy now let’s get back to the Microsoft Excel and try to implement some conditional formatting now we are on the Excel worksheet this particular data set is based on a store now we have various columns in this particular data set we have r ID order ID customer name Etc and finally we have quantity discount and profits now let’s go to profits uh let us imagine that we have provided this particular store with a target of minimum 15% profits now let us find out the stores that have hit the 15% Target and the ones which did not so for that let us select the entire column now you can find the conditional formatting option in the home toolbar and in that ribbon the ribbon with Styles has the conditional formatting option click on that and you have various options over here highlighting the cells top bottom rules datab bars data scales and icons so data bars are actually the bar graphs which highlight themselves on each and every cell if you implement them and these are the color scales if you want to implement some colors you can go through that and if you want to represent your C with icons you can also do that for now let’s highlight the sales so select that and inside this select the greater than option and here you can provide the percentage that is 15% and the sales or the stores which hit the target should be highlighted with green color and select okay and there you go you have your stores that hit the 15% Target and the ones which did not hit the 15% Target now if

    we want to highlight the ones which did not hit the 15% Target you can highlight them with the color red so conditional formatting highlight the Cs less than 15% with color red and text dark and there you go it’s done now let us consider another example so this particular example is based on train data set and here we have the passenger ID survived passenger class name Etc now we have a tra an accident incident over here now one is indicated that they are alive and zero is indicated that they are not alive now let us use this column and conditional formatting highlight the cells containing the text as one with color green okay and highlight the cells that contain text zero as r that indicates they are not alive and let us select the icons now we have the icons over here the X indicates that they’re not alive and the Green Tech Mark indicates that they’re alive now that is how you use conditional formatting in Excel Excel now here on my spreadsheet you can see some data so this particular data has some color markings so we have used color codes to Mark the designations of all the employees with different colors so for example blue for manager purple for senior and so on right now our duty is to count the total number of emplo present in a company based on the color code this can be done using filters but let’s try with color codes now we don’t have a readily available function in Excel to count cells based on their colors but we can make use of macro now to make use of macro you might want to enable the developer options so by default these will be disabled in Excel to enable them right click on the toolbar and you can see this particular option click on the customize ribbon option and you will be provided with various set of options now on the right side you can see developer options so make sure the checkbox is clicked to enable developer options and press on okay to completely enable developer window right now when you click on developer options you can see various options here visual Basics macro Etc now click on the macro to create a macro now just give a name to the macro function so I’ll give simply long and press on create to create that particular macro function so we have already created a macro that is Count colored cells which we will be using in our current spreadsheet for counting the colored cells now let us erase the newly created function now getting back to the original code so how does this particular count colored cells works so let us understand the function first then we will try to make use of it in the spreadsheet so the function name is Count colored cells as you can see and this particular function will have two parameters current cell and spreadsheet area so the current cell will be the cell address where the color you want to count is present and then the spreadsheet area so what is the range of cells where you want to locate that particular color and count the number of repetition that is the spreadsheet area and then comes the variables used colored cell range color code and color cell count so the colored cell which particular cell is having that particular color and what color code is involved in that colored cell and colored cell count so what is the total count what is the total number of colored cells you have in that particular range that is colored cell count now let us consider color code is equals to current cell interior color that is current cell do interior do color so this is the function which will identify the interior color of a selected cell let us imagine that initially color code is equals to the current cell’s color code okay now we will have the for Loop and that for Loop will include colored cell which is the current cell in the spreadsheet area and if the colored cell the currently selected colored cell do interior do color is equals to the color code that we have set selected then colored cell count the variable which is used to count the number of colored cells is equals to + one initially by default it will be zero in case if this particular if condition is true that is the current cell color and the color code given is equals to same then the value of colored cell will increase by one and this particular if Loop will run as many number of times the condition is true so let us imagine that the condition will remain true for five times then we have obtained five similarly colored cells in the given range of cells in a spreadsheet and once the condition fails then the latest value present in color cell count variable will be displayed which will be a final result so this is how the macro function works now let’s close the macro window and get back to a spreadsheet now let us try to make use of the macro function that we just created so that is equals to count colored cells press tab to select the macro function and the first one is the current cell so this is the current cell where the current color is being provided comma this cell range so our cell range will be from C2 to c31 right so this is the range where we want to count the similarly colored cells that is sky blue colored cells close the function and press enter now you can see the function has identified six repetition of sky blue color in this particular range of cells now you can drag the same formula across all the cells and it will be providing the results of the same so six senior colors that is pink or purple color then two repetition of this particular color two repetition of trainy color five for dark blue and six for green right so this is how the macro function works and the key point to remember here is this particular macro will work only for manually colored cells so there are situations where we have used conditional formatting to color a single cell let us imagine that we have used conditional formatting to recognize the employees with salary above 30,000 as green color and if we try to use that particular green color to be counted by the color count sales function no it will not happen it will not consider that color as a feedback or an input now in future we will also try to design a macro which can recognize the colors made by conditional format as well but this particular function will be exclusively used for manually colored cells only dated IF function is majorly used to find the differences between two individual dates so it is also called as dated if by a few people and a few people also reference it to date diff that is date difference so anything is good now to find the difference between any two dates let’s select a few dates so the first date would be uh let us imagine that you’re an employee and you want to to find out how many years that you’ve been working with an organization so for that you might want to need today and the date where you join the company right so it might be date of joining and today so what’s the difference right so let us imagine that you join the organization somewhere around 2010 01 05 2010 and today so for today can you can give any particular date let us imagine that uh today is 1st of January 2021 or 2022 now we have up both the dates that is the joining date and the current date which can be the current date so the day can be anything so you might be watching this on a different date as well so the date of that day for today would be a little different so we’ll imagine the today’s State as an imaginary date now we might want to find the difference that is in terms of years so you wanted to find the difference in terms of ear so here you can find the difference in terms of ear days and months as well so currently let’s try to find out the difference in terms of years so for years you might want to specify the third parameter in the data function as why if you wanted months you have to give the M as the third parameter and if you wanted to find the days you might want to give d as the third parameter uh so we have discussed the third parameter so what are the first two parameters right so the first two parameters are the first date and the last date so the first date in this situation is your date of joining and the last date or the final date which you want here as your today so let’s find the difference so for that equals 2 so it can be dated if or date diff so date if and the first state which is the A2 and comma the last state that is B2 and your parameter third parameter in terms of ears that is why close the bracket and press enter so there you go you have your date here so it’s been like 11 years that you’re working with your organization so that’s how you find out the differences so we have 11 years because uh we did not finish the 12th year completely so if we had given the month as uh maybe six where we have finished our total years then it will automatically change to 12 years so now for this particular task we will be considering the student database now let’s get into practical mode and start our Excel so on my screen you can see some data related to students now let us try to duplicate some of the rows here so we have selected the rows 8 9 10 and 11 now we shall copy them and paste in so now these are the duplicate elements right now they there might be possibilities about the duplication of class because all the students all the 10 students are in same class so there might be a duplication but we are not looking for such kind of duplications right we are looking for the duplication of the entire row for example we have the details of mic over here in the eighth row which has the role number or serial number as seven name as mic role number class blood group and subject scores percentage round of percentage and total marks so the entire eighth row is been copied here and it is in the row number 12 we have the exact same details so what if you have the exact same rows duplicated so that’s what we’re going to deal with we’re going to eliminate the duplicate rowes all completely together now to carry over this task we have to select the data from toolbar so you have various options file home insert Etc so you need to select the data and inside data toolbar we have the ribbon and inside ribbon you have got the duplicate values or remove duplicates uh option in the ribbon so select all the data select remove duplicates and now you can see that we have all our uh column names over here and make sure that you have selected my data has headers and then select okay and now you can see that Excel has automatically removed four duplicate values so four duplicate values are found and removed 10 unique values remain now another type of uh remove duplicates can also be done so you can see that here we have another data set which has phone numbers or phone names that is phone makers that is Samsung Huawei VI Oppo Etc so there are some duplicate values in this I’ve copy pasted them so now you can also do that so just select all the data and now go into home and select conditional formatting and here select highlight a cell rules and inside that you have duplicate values selecting this option will help you identify the duplicate values in your data set but not to remove them so you can select that and you can see the duplicate values are highlighted over here in the red color now again you can get back to data and uh select the option of remove duplicates and select okay and the duplicate values will be automatically eliminated so that’s how you delete or eliminate duplicate values in uh data set and you can also see that we have another duplicate in this particular row and if you wanted to select a particular row and eliminate the duplicates you can also do that now for that you need to go to the same process remove duplicates and here Microsoft Excel is asking you if you had to expand the selection to the entire data set or you want to continue with the selected data set itself or selected column itself so for that you need to select the continue with the current selection and remove duplicates and press okay and you can see one duplicate value was found and it has been removed and six unique values remain so that’s how you use the function of removing duplicates and Exel so what is sumers and Exel now you might want to calculate uh the total sum of sales right so let us assume that this is your entire stores sales data and you wanted to calculate the entire sum of sales so there is a sum function in Excel where you just have to you know select all the sales and apply some function onto it and you’ll get the sum of all the sales right so with something like this you’ll select all the cells and then you’ll apply a sums function and you’ll get the sum here and average everything right so this is fine but what if you just wanted to find the sales that are happening in only western region as you can see our column f is based on region so we have four regions South West East and Central so your manager asks you to find the sales happening in suppose say west region so how do you calculate it would you manually go into each cell and check if it’s West and Cal that will be time consuming right so what if you had a method which could do it for you just in a matter of few steps or few clicks right so that’s exactly sus so it will basically add a condition calculate the sum of sales where the region is equals to West simple to like or similar to seal query right so that is exactly what we’re going to do in Excel today that is using summs now let me add here Sales of West Region now in the next cell let’s add uh sums function remember there are multiple sum functions in Excel so when you type in suum you can see so many options based on some right you just have some followed by that you have some if it is just some if not some ifs today we are concentrating on some ifs so not just some if we’ll go to some ifs so we’ll be having an option of placing multiple if conditions there so let’s also try that more on that later now let’s press tab to select our sum ifs function now okay we forgot another important step so before we apply sufs we need to convert our table into a table format yes it looks table but right now it’s not in Tabo format it is in the format of database we know that Microsoft Excel considers its data as a database by default so to convert the database into table just click anywhere on the data and press contrl T so that you’ll have a new dialogue box and and it will ask if your table has headers just make sure that you have Ted in it our table does have headers so select okay now our database has been converted into a perfect table and it’s ready for applying sums function so now you can see our database has been successfully converted into our table now let’s begin with our sum ifs now again remember we are using some ifs here not some if or sum so select Su if and press Tab and we have our sus function now the first step which column are we looking at what are we finding out right which is a sum range what sum do you want for now we need sum of sales so select the sales column and we have a simple key function here let’s hold control again press shift and hold shift together so we’re holding control and shift key together and now we’re pressing lower Arrow key once yes the entire sales column has been selected comma Now where do you want to look so we want to look I mean which part of the table you want to fetch right so this is the region part we want to fetch right now so again hold control and shift together lower Arrow key the entire column is selected now the next last part so we want to look into the both of the columns that is sales column and region column now which type of data you want so for now we want Western type of data that is the sales data which is happening only in West right so we’ll select this cell with the data as West or you can also manually enter it for now let’s select the data so I’m selecting the cell F7 which has uh data as West right now let’s close the parenthesis and press enter so there you go you have the sales happening in the western region now you can also try to format it in the form of currency so let’s use uh dollars since it is based on American states so let’s use dollars so there you go now we are having the sales data of West now let’s try to copy the same to all the for Cs and uh now here let’s type in East and this can be South right and this can be your central there you go right now you can also copy the same formula and make minor edits to it so here instead of f8 you can select uh East so where is East here and press enter so you have the sales related to East now let’s try to edit F9 with South select South Region enter so you have South sales and what remaining is Central so let’s edit Central as well and press enter so there you go right now you have all the separate sales data from West west east south and Central now again uh you might want a bit more granularity in data right or in your reports right like you wanted to find the sales data happening in West and you wanted to find out the category based on only Furnitures so your manager will ask you okay a fantastic job so now just give me another minute detail on the data I want to calculate the sales happening on West based on category Furniture right so he wants furniture or he wants office supplies only right you can also do that it’s not an issue at all now let’s try to copy the cell and paste it here and sales invest let’s add another input here on furniture right now you wanted to calculate the sales happening in west region only on furniture so you can also do that now we can use the same formula as well just copy paste the formula here right and you can add few things here when you try to add a comma it will automatically generate the next criteria so it is expecting that you are looking for adding another criteria right now which column you want you want category so let’s select category you want to select all the cells same trick control shift lower key and again press comma and you want it to select only uh Furniture so select on furniture cell there you go now press enter so there you go we successfully have the sales data from west region based on category Only Furniture so that’s how you do su and Exel you can also add a few more details to it like in Furniture you can also look only for tables so here you can see in west region we have furniture sales data and we have sales data of West Region based on furniture which includes only tables you can also do that using sums function in the same way all you have to do is Select this subcategory column and select the cell which has the data as tables so this is how you use Su in Excel now creating a pivot table is just a way from a few clicks trust me it’s just a couple of clicks and you’ll have your pivot table ready to operate now click any cell or select any cell on your spreadsheet which has the data and navigate to insert menu on the toolbar and select pivot table and that’s how simple it is now automatically Excel will choose your data you can see the line L over here dotted lines that means the Excel has already selected the range of data that you want to put into your pivot table and you can either select the existing worksheet or a new worksheet I’ll select a new worksheet over here and just press on okay and there you go you have your pivot table ready now here you can see your pivot table is a little empty all you need to do is drag and drop the data elements that you want on your pivot table so here you can see on our spreadsheet we have some business data related to our store which has has furniture and states and subcategories of the furniture and all the you know in in category you have furniture office supplies technology Etc right so those have some subcategories as well and different states and the regions as well and the quantity total sales happen right so using this business data we can find out some key insights of this particular data using our pivot table right so here we’ll add the data now let me drag the region into rows click and drag and let me drag uh categories into columns which are the different categories which are involved in the sales data and values what are the sales happened right and there you go you have Furnitures office supplies technology as the three different categories and those are the sales happened in central East South and West regions and this is the grand total and instead of regions you know you have all the regions over here right right so Central East Southwest or instead of uh all the regions or instead of all the categories you can you know place a specific category in your pivot table for example let me push the category into the filters and here you can see the total sales happened in those particular regions now what if someone asks you to find the sales of Only Furniture right so your manager comes to you and he’ll ask you getting the sales of only Furnitures you can select that and there you go you have the furniture sales in all the regions right or you can also do the vice versa you can also drag uh regions into the filters and if you wanted to find out the region wise sales of furniture you can also do that right so here we have furniture and here you have all the regions and let us try to find the sales of furniture in the central region and there you go right so this is how um the priv table works and you can also remove the regions for example if you wanted to remove a certain element from filters or anywhere if it is columns rows anything you can just drag and drop it right so that’s how it works and also you can increase the level of detail or improve the granularity right so here we have U categories on our uh you know columns so you can also include subcategories into the column right and and you have the region wise sales so in the region filter select all press okay and in the category select all press okay and you have the furniture here office supplies here and Technology here and also if you want to include the quantity how much or how many number of um you know things got sold in those particular categories and subcategories you can also do that so here in the furniture section books these are the sales and these are the number of quantity or elements sold in in those particular you know sectors so you can you can also improve the level of granularity you can also improve the level of detail using pivot tables in now AutoFill in Excel is a feature where you can add some series automatically using Excel for example let us imagine that you wanted to add index numbers right so starting from 1 to 10 so manually you will be typing in the numbers and you’ll be doing it right so instead of this what if I said you there was a simple way where Excel will fill all the data for you right so just start with one two and select these two cells and you can see the flash logo over here right so that’s the indication that Excel has understood your pattern and it is ready to fill the remaining numbers so as long as you drag this logo or that ending part where you have a small square box that will fill all the numbers until where you have draged it so that’s how autofill works and you might be wondering will it work only for numbers no it will also work on some Logics as well okay this was the first part let us also look at some other tricks with autofill now most of the time time you might be just adding one number and try to drag that cell and the result is it’ll copy the same number all the time right so you can change it by clicking on this menu you can click on fill series and it will change to series and this was one more trick with numbers now getting back to the next type so let us imagine now the next type of series you want to add is weeks in a day right so you’ll be having Monday Tuesday save and stay right so you can also do the series of weekdays using AutoFill in Excel as well so drag that cell and it will give you all the days or week days in a week now let’s try January and this will give you the months in a year in the shortcut TOA right since you just added the first three letters of the month it will recognize the pattern and it will do the same for you now let us try the full names of months right and now let’s try to drag it it’ll fill all the full names of all the months now let’s try the dates so let us imagine that this is the first day of the current year right it will now give you all the dates of that current month or until the end of the year as well there you go now let us imagine that your manager tells you to add the dates of working days in the month of January so how can you do that it’s simple all you need to do is select the cell go to the fill option select series and here select column wise since you w the dates and columns and here in the type select date date unit should be weekday and the stop value should be the next month right since your manager asked for the weekday of January so you stop value will be the 1st of February press okay and there you go so since the 1st of January is Friday it’s selected over here and the rest two Saturday and Sunday are eliminated similarly we have all the dates here of working days only so that’s how you use AutoFill in exit so on my screen you can see a simple spreadsheet with student names and their marks on my sheet so we will be calculating what are the total marks obtained by the students in overall subjects and calculate what is the percentage now we will carry forward a few simple inbuilt functions in Excel to calculate that first we will be calculating the sum of marks so here equals to sum function and select the range of cells close the bracket press enter so you have the sum now so we have to calculate it repeat so by clicking on the the same formula now we have the total of all the subjects for individual students and now let’s calculate percentage so usually we use the you know divide by option that is divide the total number of marks obtained by the total number of marks available that is we will be dividing 47 by 500 and then we will be multiplying that into 100 so that’s how we calculate the percentage so cell G2 divide by cell H2 press enter and you will end up with you know uh the decimal format that’s perfectly all right now we have also extended the same formula for all our cells now select this overall column or select the highlighted cells right and get into the data type here you have General so that’s why we have got that uh decimal format but when you press percentage you’ll have the actual percentage of all the students according to their marks achieved now we are on our Excel spreadsheet and this spreadsheet has some values of employees that is their employee ID name designation Department salary date of birth and date of joining now our concern is to highlight some cells which are duplicate in this or some rowes which are duplicate so for example here you can see on rule number seven we have the details of Mara and you can see the same details of MAA in the row number 14 as well right so you need to eliminate those cells or you need to eliminate those rows now you can say there is a quick way just few clicks and those will be eliminated that is conditional formatting yes that is also one way but I’ll show you what’s the problem with that okay so let us select the entire table and get into conditional formatting and here you have duplicate values highlight cell rules with duplicate values and there you go the problem so here we have designation which is duplicated in certain ways department and salary state of joining right so there are certain conditions or scenarios where you cannot directly identify which is the duplicate rowle again you can use the same for this right so you can select the entire uh column of employee ID and you can uh identify the duplicate cells here since the employee IDs are unique for every one so this is a way but let’s dig a little deeper and understand the way of highlighting duplicates with an advanced technique so let me cancel this and for that let’s create a new um column and let’s name this as duplicate flag e flag so the new column has been created which is D flag so here we will be using a simple formula that is Count F and here press tab to select the formula and here we will be adding our range that is which range are you considering so the range will be the employee ID since employee IDs are unique for every employee and and let’s press function F4 to fix it and now uh again the criteria and the criteria will be the same cell that is A2 and now close the bracket and press enter now let’s track the same formula to all the rows okay now here we have now what you can do is uh sort it in an advanced way go to sort and filter and custom sort and here you’ll be sorting by the D flag and it will be smallest to largest so now uh you can apply the conditional formatting to highlighted you can select the column H or the D flag go to conditional formatting highlight cell rout and here you can get into greater than and give the value as one okay so all the cells which are you know repeated multiple times will be selected here and they will be highlighted with a selected color what you have given so I’ve given Red so now you can select these cells and highlight them or delete them from your spreadsheet and now all your data in the spreadsheet is unique and clean now let’s quickly get onto the Practical mode where we will have some data sets over which we will be trying to apply the charts now there are various types of charts in T and those are the pie chart the column chart the bar chart column versus line P chart and sparkline charts so we will discuss one after the other first let us discuss about the pie chart so as you can see we have a small data set on my screen which I have personally created and uh this deals with the companies and this deals with the shares of the automobile industry so now for this let’s create a pie chart select all the data and then get into the toolbar and inside the toolbar you can see various options and inside that we going to use the insert option and when you select the insert option you will have a ribbon here and inside the ribbon you can see groups tables illustrations addin and here you go these are the charts so now we needed a pie chart so select on that icon and you have various types of pie charts you have two dimensional pie charts three dimensional pie charts donut type of charts Etc right so I would like to go with the three-dimensional pie chart as it looks a little more appealing and easy to understand and now there you go this is the title of the chart and these are the legends of the chart and this is the chart area and these are the values now we can uh know kind of add the data labels as well so you can just have to press the plus icon over here and you’ll have the options you can add the data labels which will give you a lot more interesting values and uh you can also turn this around as you can see we on double clicking any part of the pie chart you’ll get this option which is called as format data point and uh let’s imagine that you wanted to show the information related to this particular one which is Volvo which has the biggest share so this is in front right now now what if you wanted to show the information related to data which is over here right for that you can select this particular Arrow key and you can just twist it and there you go it is coming in front this will be uh a great way of presenting your data to your clients right yeah and there is another option where you can directly explore this like you you can pick it out you can see this right Point explosion where you can split it out and you can show that especially like you can highlight it and show it so that’s the you know a great way of presenting your data so that’s how you can work with pie charts in Excel now let’s quickly move to the next one that is the column chart and in the column chart we have the data related to some company and uh the profits of that company based on a so you can see that the year has started from 2008 and it has gone until 2021 and these are the profits of that company year after year so since we are dealing with column chat I’ll tell you the simplest way to create a column chat there is a shortcut key for that which is alt F1 and there you go you have the chart right on your screen now you can uh expand the chart just by dragging it like that and you can see that there is some problem with this chart that is we have numbers here instead of ear right you can see that the First Column is the es which is 2008 to 2021 but here you can see we have 1 2 3 4 which is not proper right so to change the axis that is you can see the axis which is 1 2 3 let me expand my screen a little bit Yeah so if you look at my screen we have two different columns one column is about the ears and the second column is about the revenue now what we want to do is we want to create a line graph where we have ear on the x-axis and revenue on the y- axis and we will be generating a line graph based on the data we have now for that we have to select the entire data what we have then we need to go into the insert option in the toolbar and then you can can see a lot of chart options in the insert ribbon now here you can see that we have a line chart or area chart option select that and here you can see 2D line graphs now let’s select the first one and there you go you have the line chart here now let’s rename the title and there is one problem with the chart here you can see that the x-axis is not actually pointing the ears you can make a change to that no worries just right click and then you can see the select data option and there you go you have the edit option here just edit it and now select the range I am selecting the ears now let’s select okay and there you go you have the years here you can just try to expand it a little more yeah so now you have the revenue on the Y AIS and year on the x-axis and this shows the growth of Revenue in your company so this is how you create a line graph in Excel C on now let us imagine that you are the CEO of an IT company and you have taken up multiple projects let us imagine that you wish to keep track of all the projects so what if there was a tool which could keep track of all the projects in the form of a detailed visualization with the percentage in the form of a graph or a bar chart interesting right that’s exactly what we going to design today using Microsoft Excel so without further Ado let’s get started with our onepoint agenda that is progress tracker in Excel now to design a progress tracker in Excel let us get back to the Practical mode where we might have to use our Microsoft Excel now we are on the Microsoft Excel and we have taken some sample data to make things look simple so here we’re considering a set of students and their attendance percentage so we will be using this particular data and design a progress tracker now let us try to increase the size of the cells now select all the cells in the attendance column now we have selected all the cells in the attendance column and in the toolbar select the Home tab and inside the Home tab you have styles group inside the Styles group select conditional formatting and inside the conditional formatting option select new rule so so in the new rule we want to select the first option that is format all cells based on the values now here you can see format style since we trying to create a databas we need to select the option of databas here and under the datab bar options we have some more options on the drop- down menu so here in the type select number and the minimum number is zero it is automatically stored here and the maximum value is again a number and let us provide it as one now here you can select the color for your pass now since we have already chosen green for all the column headers let’s choose blue and select okay now here you can see that the values are being indicated using bar graphs right and you can also see the percentage values but we can make it look a little more better by shifting them to the left side and text could be given as white so that it’s more uh vibrant to look at and now we can also apply some conditional formatting onto these attendance columns so let us try TR to highlight the students having attendance less than 65% so highlight cell rules less than 65% attendance with red color select okay and there you go you have the progress tracker right on your screen and and using the same principles you can track the attendance for all the students and also you can apply some conditional formatting onto all your cells and highlight the ones which are not on the track now this is how you can Implement progress trackers using Excel now for this particular example we going to use the employees data now let’s quickly get back to the Practical mode and enter into Excel now you can see that we are on Excel sheet right now so let me expand my screen a little bit so that the data is a little more visible yeah so here on my screen you can see that we have employees data so the First Column is the name of the employees and the second column is the start dates of the employees that when have they started working with the organization and when they have ended or resigned to their position in that particular organization and the duration gives us the number of days they have worked in that particular organization so to create a g chart is really simple you just have to select the data and then go to the insert option and in there you can see the charts icon and select all charts and in that select the bar chart and here you can see the Stacked bar chart select that and press okay and there you go you have the Stacked bar chart right on your screen let’s expand this a little more so that we have the clear visualization of the dates as well now you can if you want you can change the title as or RG data or employe data and now we are partially done with our chart now the next thing we have to do is select the bar chart right click on it select data and here select the add icon now you need to select the series name so the series name is duration for our gchart and series values are let’s select this and select let’s move a little yeah so these are the data series let’s select okay and you can see the Gan chat on my screen right here select okay and it’s done and still now it’s not complete yet so to finish it you need to select the blue colored bars here and right click on them and select format data series go to the fill icon and select no fill and it’s done for now and you can see that there is a difference in this particular arrangement we had Joe at the first place and Susan at the last place but here you can see Joe is in the last place and Susan is in the first place so we need to make some arrangements here for that right click select format access and then here you can see categorize in reverse order and there you go it should be done and there you go you have your Gant chart right on your screen so window here we have some data set so we have row ID order ID order date Shi date Etc so these are the rows over here let’s quickly select the entire data set and remove any filters that we have so the filters is gone and uh at the last you can see country region product category subcategory and extending further we have sales data quantity ordered discount and profit so now let’s proceed and try to build one single dashboard okay now imagine yourself in a situation where you’re working for this particular you know um store or a company which sells products and you have a team meeting with leadership and you are supposed to be hunting for simplicities in complexities and you need to present an interactive dashboard which gives a visual representation of what exactly is happening with the company the sales the quantity order profit Etc right and if you show this particular data set as it is in the form of named rows and columns your leadership might not be able to understand it or they might not be able to justify what exactly needs to be done in the future to keep the numbers same or improve the sales right so in that scenario an interactive dashboard could be beneficial so in today’s session we’re going to create the scene and creating a chart that is an interactive chart or a dashboard which is interactive everything lies on one single point that is the pivot tables and pivot charts if you’re good with pivot tables and pivot charts you can create interactive charts you can create dashboards and much more so the process is simple but in case if you need a quick overview of how exactly a pivot table is created or performed then you can check out the link in the description box below which will give you a complete detailed description and to understand how to create a pivot table and a pivot chart in real time and move further right now we will continue with this particular topic for today which is creating a dashboard now click any where in the cell select the entire data set using control a and go to insert option over here and choose pivot table right from the table range should be the option here you can see the table range has been selected and uh you also have two more options which will help you understand where exactly to create your PIV table so we will be creating that in a new worksheet so click on a new worksheet and press okay there you go you have the Pu you know table here and let’s say we want a few charts so here we have a region wise sales country wise sales and shipment majorly used subcategory uh category segment and let’s say we also want quantity wise country and quantity right or let’s say Okay country and quantity so how many number of orders basically you receive from each country right so let’s say we want all these charts now how many number of charts do we have 1 2 3 4 5 6 and seven remember creating a chart which is interactive relies on pivot table now so far we have created one pivot table or one pivot chart or one pivot sheet but we want seven pivot charts how to do that it’s really simple just hold control and drag that sheet and you have a new pivot chart right you can do that few more times seven so we have seven pivot tables as we required there you go and now let’s get back to the first sheet and let’s see the first option we looking for so we want region wise sales right we have five different regions north south east west and Central and we want region y SS in a pictorial way for that let’s go to the options over here we have the row uh names or the metadata you can drag region into either columns or rows wherever you want I’ll Cho and just drag and drop it in the columns and now go back to sales and you can drag sales to rows values sorry sales in h region right now you can go back to this particular pivot table go to home home eliminate any and go to pivot table analyze and here you can see pivot chart click on that and now you have a all charts window on your uh desktop I mean screen you can choose the one which you uh feel is good right so I’ll prefer a bar chart column chart here click on okay and there you go now you can edit this particular data set and chart and you can also remove any of the things you don’t want so for for example I don’t want Legends so I’ll remove Legends and uh this is what I have here you can also edit for few more things right now I can simply copy this particular chart and go to the dashboard so let’s create a new sheet and name it as dashboard merg and Center can add a color Bo now here you can paste the first chart now you can rename this chart title so you can rename this as region wise saes so therefore you have the first chart we here you can also increase or decrease the size of the chart by just dragging the corner points and uh let’s go go to the second pivot table we have the second pivot table right here what was our second request the second request was country wise right now close this the country and drag it to columns or rows let’s go with rows this time and SES go go to pivot table analyze pivot table chart and here you can choose a pie chart since you’re going with the country wise and click okay now you have the by chart over here you can rename that as country wise you can copy this and paste it onto the dashboard in case you want to you know uh keep the size of the pie chart and the column chart in the same uh you know width and height you can do that just go to uh format here you can 5.6 and it can be 9.9 and similarly you can go with 5.6 this can be 9.9 just so that you can accommodate a few more chats in the uh dashboard you can align them like right now our next request test is sub category shipment sorry so go to shipment so this is the fifth uh sheet so we need to go to the third spreadsheet where where is where we have the pivot table so close the format shape and we have shipment here ship mode so select the ship mode drag it to rows or columns Sal to here and we have uh or you can count it right you can change it to count okay so you have standard class with the highest number of um you know sales happening so many sales which are happening in your store choose the standard class of shipment right so very least people choose first class where or same day shipment which is highly expensive and standard shipment will take minimum pricing and most of the customers have chosen that now you can choose the same data here go to pivot table analyze and here we have pivot charts and of this you can select anyone let’s say if you want to go with the tree map or if you want to go with the bar or anyone you can go with that so let’s go with the bar graph press okay and here you created now rename this particular chart as ship mode now now you can copy the same chart and press paste it onto the dashboard can close that if you go to format you can do the same setting 5.9 and 9.9 4.6 and 9.9 there you go you have everything lined up and this space that we have over here we can add a few more things which will come under slicers and filter do so we’ll have to wait for that until we finish the rest of the charts now we will go to the fourth sheet so what was the fourth request let’s check our list here so subcategory wise sales now we can check the category subcategories over here yeah we have the subcategory select that and track to rows or columns and sales to values okay and there you go and if you want to sort you can also sort by right clicking and sort it to minimum smallest lar largest to smallest there you go so the highest number of sales are happening through mobile phones next comes the book cases next come copers etc etc now select this and go to pivot table analyze pivot chart and you can choose any one of those we can go bar chart or the column chart and you can can also rename this as subcategory device you can copy it based it on the dashboard and you can also make few Arrangements if you want you can resize it 5.6 and this will be 9.9 the next request is about the category wi sales and segment so we have the category here drag it to row and sales here you can keep it the way it is or you can go to Pivot analyze and here go to charts choose the one you want and proceed copy it and go to dashboard you can paste it over here there you go country wise and segment quantity so now you we have country over here country to rows and sales to columns or you can yeah let’s go with sales since this is a sales dashboard this should be beneficial now we have uh we can also sort it again sort to largest to smallest and we have France with the highest number of sales I can copy that or go to pivot table analyze and here pivot chart we can go with by chart you can rename it as country wise there you go and copy this paste it on the dashboard I think we already did a country wise sales so I think we can proceed with the next type of uh chart which is segment wise let’s go to the same sheet where we created the country wise sales and close the format delete the chart instead of country we can select segment so which segment has the highest number of sales past it over here format to adjust the size of the chart there you go so we have region wise sales country-wise sales ship mode subcategory wise category wise and segment wise sales now we have a dashboard but to make it a little more interactive we might require a few filter like options which are also called as slices so when you add slices onto the sheet or the dashboard what you can do is you can select the options you want let’s say I want the region wise sales in Belgium then you can identify that let’s say you want Australia’s or Austria’s shipment modes what kind of shipment mode is most used in Austria you can you can find out it and let’s say you want to find out the Finland segment wise sales then you can find out this is only possible when you try to include a slicer into the dashboard now let’s go ahead with it now if you select any of the charts let’s say I want to go with the first one now go to insert and here you can see filter options right so here you want to insert slices now here you have a slices right now okay so you can select the slicer you want so I want the slicer for region I want the slicer for Country I want the slicer for ship mode which is right here and I want subcategory wise slicer I want country or category wise slicer so category and segment wise which is here so I want six slices there you go I’ve got the six slices created let’s minimize to make it look a little more better there you go and uh we will paste the slices right here you can also you know modify the size of the slices let’s try to make them fit here and uh the next one should be right here there you go now let’s drag this here the last Slicer in the last section there you go now we have an dashboard but is it interactive I guess not so how to make it completely interactive so you might have a doubt let’s say I want to click on machins now only one out of the six charts is you know uh reacting let’s say I want to click on papers it is giving me the data for phones and Etc and if I click for Belgium you can see only one chot is being interactive right now what should I do so that every chart which is on the dashboard gets completely interactive and shows all the data in real time that can be done all you have to do is select any of the uh slices right click and go to the option which says report connections so select all the pivot tables pivot table which are present in your uh chart right now sheet one sheet 2 sheet three sheet four sheet Five sheet six and Sheet 7even press okay now all the chart cards are interconnected with each other and now if you click on any one of the options let’s say I want the data for Austria country and every data changes correct report connections you need to do that with all the slices press okay there you go now if you select Finland Denmark or Austria you can see the charts are changing right every chart is now interactive so that’s exactly how you create an interactive dashboard now you might be having a doubt okay it’s working for the existing data but what if I add some new data so here we have data until 2022 and this is the new data until 2023 now if I add this particular data to the end of this particular data set right here and remove the row heads and go back to dashboard go to Pivot chart yeah on data dashboard we need to go to data and here click on refresh all then you will get the updated data set and now you have the updated results as well and with that we have come to an end of the session on sales dashboard in Excel or realtime interactive dashboard using Excel in today’s video we are diving into Power pivot in Excel so if not aware power pivot is a powerful tool in Excel that allows you to import analiz large data sets from multiple sources for example if you have any separate data sets for sales transactions and product details the power pivot enables you to link them together using a common identifier like the product ID create custom calculations with Dax like the total or average sales and quickly generate interactive reports like the pivot tables basically it simplifies the complex data analysis making it easier to work with the large amounts of data without relying on traditional formulas and help users create meaningful insights in a more efficient and error-free way by the end of this video you will be equipped with everything you need to master power pivot in this video we will cover activating the power of pyo as an Excel addin creating relationships between tables calculating the profit total profit and more and wrapping it up with kpis to filter data like a pro this video is packed with exciting and super informative tips to level up your Excel game so I am inside Excel and before going to the pivot table I have a list of data over here I have basically three tables the salaries transaction and the employee and we can see that some columns is common like the employee level the employee level over here and in the salar in the salary Street and the level column and the employee sheet they are the same so let’s get started with installing power pivot for that we need to go to file and then options and here in the addin as I have already activ ated it so it is in my active application addin you will find it over here in the activate application addin if you have not activated and you can just click over here and in manage Excel addin you can go for com addin as it is a type of com adding and then click on okay so I am inside Excel and before going to the pivot table I have a list of data over here I have basically three tables the salaries transactions and the employee table and we can see that we have some columns in common like the employee label in the salaries table and in the employee table we have a level section these are basically the same so let’s get started with installing power pivot for that go to file and then options and here in the options then you have to go to this addin and select Microsoft power pivot for Excel it is in my active application addin as I have already activated it you will find it in an inactivated application addin if you have not activated it so you have to just click on it and in spite of excel addin you have to go for com addin as it is a type of com addin and then click on go and when you click on go you will have a drop down over here for in that drop down you have to select Microsoft power pivot for Excel and just click on okay and there up in this section you can see this power pivot just got available and now we can close the file and open a new Excel file where we will do the calculations and Analysis a blank workbook so we need to cross this and in the new Excel file you can click on power pivot and then click on manage and this will take you to the power pivot in Excel sheet right now we don’t have any data so let’s import data from the external sources so let’s go to external data from other sources and you just need to scroll down and you will find the Excel file over here and then click on next and when you click on next you will get this Excel file path basically you need to locate the place where you have already saved your Excel file so click on browse and as I have already kept in it my downloads I will just open this I will take this box as use first row as column headers and click click on next and now you will see the three available tables over here and I want to import all the three and hit on finish after it is being initiated you can close this and when you close this now we can see everything is uploaded over here we have employees salaries and transaction now that we have whole data over here you can do the diagram View and see let’s go back to data View and try to delete this delete yeah now we can go to this diagram View and let’s build some relationships we can now basically connect staff ID over here with that staff ID of the transaction table so you can just click over here and attached to this so it has created a relationship same goes for the employee level the employee Level under the salaries table and this Level under this employee table we’ll connect this so it got connected so now that we have basically added connection it’s no longer just three tables but one large data model now we will go back to this data View and we can basically add columns if we want as we have the selling price and the cost price we will find out the profit in add a column you need to write here just equals you have to just click the selling price minus this cost price and just click enter over here this is basically our profit and now we will rename this as profit now we have the staff ID what if we want to know their names as well not only their staff IDs now basically we will write over here equals related and you will get a suggestion over here as related and then if you click on enter you will get all these options now basically I want this employee name to be written and click enter and close this parenthesis and finally click enter and see you have got the names now what about measures asume we want the sum of all the profit we all meet we will go down somewhere over here and select a sell and over here I will write equals sum sum and click on enter and then choose profits and I will close this parenthesis and put on enter see it’s done now I can rename this as total profit as it looks good and click on enter definitely it will change here in the cell as well now we are all set to begin the analysis and click on just pivot table and then click on existing worksheet and just okay it may seem like a normal pivot table but it’s not we actually have three tables over here employees salaries and transactions let’s assume we want to see which employee is getting what Revenue so under employees we will drag the names to rows we will drag the names to rows and we want the revenue that will be under the transaction table selling price and I will drag you to values then if you want to see the level they belong to under salaries and the salaries we can often go to employee level and add it to the columns now it looks good now when you move forward just beside this measures we have kpis and click on kpi new kpi we can have an absolute value like for example I’ll give 20,000 that is a target value basically and I will choose this icon and I’ll click on okay now I will clean it a bit okay I will only have profit and values under it so under the transaction we will go to this total profit I’ll click on status and see I have got this Anna and John is basically not doing well they have basically failed to achieve the Target and the whole team may be not doing very good this Alex and Sam they are basically having a not so good one but yeah they have passed it and Jones too so Microsoft 365 co-pilot is one of the latest AI products launched by Microsoft so the Microsoft 365 co-pilot will be your AI assistant for all your office sus from Microsoft so it can help you EAS up your process of making PowerPoint presentations it can help you write down the notes on Microsoft Word and many more but today we will be using this particular 365 copilot for our data analytics using Excel so let me remind you this particular AI tool works with the versions of Microsoft Office 365 and later right now there should be a new feature added upon to your Excel on the top right corner of your home button but again if it’s not it’s completely all right you can use your Microsoft home button where you can find the Microsoft 365 Co pilot so let’s navigate to the home button yeah now I’m on my home button and here you can see a small icon which is completely new out of nowhere this is your co-pilot right and you can just click on it and you can fire your co-pilot and uh today’s point of discussion is about co-pilot with Microsoft Excel so when you are starting Excel there are some prequisites when you’re working with copilot So currently it is in the preview mode so it is capable of using some data set which is in the table format I have converted this into a table format it’s really simple contr a to select all the celles and then contrl T to convert this table uh the option my data set has headers and okay so your data it will be converted into a table and another prerequisite is when you pop up the Microsoft uh co-pilot it will asks you for a data set which is under 1,000 kilobytes so my data set here is about 800 kilobytes which should be here so the only process is you just have to click on this particular option which says attachments go to the data set that you want to work today so I’ll be selecting my sales EU copy and just press open it it will take a little time because it’s completely new so it might take a little time for that so the only uh catch is Microsoft co-pilot is a little slow compared to the other competive co-pilots or AI modules it’s not a big deal they will catch up soon let’s wait for some time while the D set loads and then you can fire some of the commands let’s say I have given my command here please help me create a regional wise sales report with this data set and can also give a few more commands right so it’s loaded right now you can see the icon here now you can write down give me country wise sales report and just fire the command you can also um you know perform all your data analytics what you regularly do right when you have a sales data set you might be wondering what it can do and what it cannot you just have to build a story about the report what you want to create create and then just you know write it down to the co-pilot and it will generate the report that you are looking for currently it is working on the country-wise sales report so it should be giving me the report shortly it’s okay so there are some problems with this so maybe it’s the data set data type so here we have uh country and the data is General so you might want to change that to text format or character format so you have the text data type here and just okay now save it back now you can ask the copilot once again okay or let’s try to do a different one okay now here we have the order date and column C right now let’s say I wanted to find out which day did they inquisit right is it a Friday is it a Saturday or Monday right so just try to ask the copilot let me write it down can you help me find out which week of the day was an order placed you we have a column named aate column is C we need a new column that tells me the day of the week when an order was placed so let’s file the command now it should give us a column D which is an empty column right next to order date or it should give us the steps to do the same just by the command it might take a little while okay here we have it certainly I’ll create a new column D and you D set great this is what we needed and so let’s try to ask another query can you tell me name of the customer with highest sale record in the data set so the previous query which we fired at will not be reflected on this particular D set but it can be reflected on the basis that shared with the go pilot here so currently I don’t have a

    direct connection with my Excel over here I think it should be taking some time for getting me an update over here but anyways it will be updated in the say that it is currently dealing with that’s not a problem now let us see what’s the answer I apologize but it seems that I’m unable to extract any text content from the uploaded file it’s okay let’s try to file there are some discrepancies but it should be resolved soon but let’s let’s try some other thing like region wise sale so let’s say you are an Excel expert so you can do it within a few fraction of seconds you can just apply filters and get that data but there are a lot of people new to excel so for them the co-pilot from Microsoft could be a boon right but down the line there are improvements rolling out so I could predict that it could be a lot faster and can have seamless integration with your Microsoft Excel and your co-pilot and things could be a lot easier right so far we have the result over here so it is giving us a few results and it is finding a little difficulty to extract a few that shouldn’t be a problem in a few couple of days where the new updates roll out so um proceeding ahead let’s try to give it another query or so so let’s check the data set so here we have the categories and sub categories I think we can give it uh okay can you give me the category wise profits so here we can identify which category is performing really good and giving us highest number of profits so I think let’s check out if we have a profits column yes we do have a profits column over here so let’s check that brilliant right so it’s giving us distilled reports in one prompt so that should be helpful for any of the data analysts which are aspiring right and which don’t have have any idea about how Excel really works okay it seems that I’m able to extract text content from uploaded files okay let’s try to copy this and write it down as okay is it category okay let’s copy this as well I think it’s about the case right category subcategory okay let’s let’s first subcategory category sales report okay it’s giving us the pre answer to the previous query okay I think this should help it’s unable to extract a lot of answers so far currently there should be some discrepancy but down the line it should be fixed able to extract any text content from the uploaded file okay so basically that’s how you can work with your co-pilot in Microsoft 365 Exel so there are some discrepancies so far so it should be fixed down the line so that’s how you can use copilot in Excel for your data analytics and and with that we have come to an end of this session on Microsoft 365 co-pilot for Excel and if you have any queries regarding any of the topics covered in this session or if you require any of the resources that we used in the session like the data set that we used which is in tabular format and under 1,000 k byes please do let us know in the comment section below and RM effect Parts we’ll be happy to resolve all your queries on the aist let us know your experience working with co-pilot and Excel and we can learn a thing or two from you as well hey everyone welcome to Simply learns YouTube channel today we will learn how to use Microsoft power query that said if these are the type of videos you’d like to watch then hit that like And subscribe buttons and the bell icon to get notified now without further delay let’s get started with the topic which is how to use Microsoft power query so imagine you have a lot of data it can be an Excel file it can be a PDF SQL database cloud or anything if it is Data it’s your data right so you have a lot of data now you have the source of the data after that I’m absolutely sure that the data is not clean enough to perform some analysis so you go through another step where you have to clean your data from unwanted rows and columns which doesn’t make sense or might have some invalid data right so you need to make sure that your data is completely accurate and has some standards for your analysis right so that’s the next step which is clean and transform after the clean and transform step you have the next step which is load that particular data to your platform it could be Excel it could be some data visualization tool like TBL power VI Etc right after that you generate some meaningful information of that you create some pivot tables pivot charts and uh line graphs Etc right so which makes the some sensor of that data you will understand what what exactly is happening with your business how many leads or how many traffic orders what’s right and which is the best performing product which is the worst performing product and you need to take some decisions on your business which can make some impact on your overall profit or Revenue right so imagine if you want to do this for an year if you get one year’s data and you do it in a day or two it’s fine and if it’s on a monthly basis you get month-on-month data and you want to do it okay it could not be worse you can spend one or two days and it’s fine but if what if it’s week on week you start to think right what if it’s a daily task it’s a mundane task right you have to do all these Steps step by step repetitively and it it doesn’t make any you interested anymore in that particular type of job and uh the major point of this is you end up spending more time on cleaning the data transforming loading it and the worst part you don’t get proper time specific time which you want to actually dedicate to plan out something and bring up strategies to improve your business right so what if you could automate this right what if there is someone who could perform all these steps for you in in the way you want and you can actually have some time to produce some pivot tables pivot charts and then understand what exactly is happening with your business and find out some critical points where you can improvise and bring up some good Revenue what if there was someone who can perform all these steps for you clean your data transform it the way you want load it to your platform and help you create your P charts and P tables and actually save you a lot of time where you can spend all that time understanding your critical points where you need attention you can fix them and plan some strategies out to bring out some good Revenue right that exact someone who can do all these steps for you and automate your data cleaning data transformation and loading tasks for you is exactly the Microsoft power query so you can use Microsoft power query either in Microsoft Excel or Microsoft PI right now let’s uh kind of generate some data so uh this data will be generated using chat GPD and we will not generate data like which is not small like for a week on WE analysis or a day-by-day analysis we will create data for years so we will take five consecutive years starting from 2020 to 2024 and we will have some uh data generated using chat GPD and we will Auto me that data and try to get some analysis done right let’s jump on to chat GPD so before we go to chat GPD I have a prompt written here so this particular prompt will guide or explain chat jpd what exactly you want right so giving some uh like the Excel data right over here we have audit date order ID shift mode and customer name customer ID everything right so we have the sales quantity discount profit Etc so in the same format or a similar format if not the same format we are explaining chat gbt that we need these columns the order ID order date delivery date Etc and we want all these files in a downloadable Excel file links right so you can also specify this column should have certain certain things and this column can have certain certain things right so we will copy this prompt and head back to chat GPD and you can just paste the prompt and and hit enter there you go now your request has been analyzed and shortly you should be receiving some downloadable Excel files or some recommendations from chat GPD I’ve already performed this process and we have already downloaded some data so there you you go you have the data from 2020 to 2024 you can just click on this to download the file so I’ve already uh done this job and U I’ve downloaded the files so so those files are exactly in my downloads here so if you go to downloads here we have ERS data for 2021 2022 2023 and 2024 so this orders data has just one file which is orders 2020 so we will perform the data cleaning so first we will specify the source of the data then we will perform the data cleaning and transformation process then we load this particular Excel 2020 data into Excel and then we load this Excel data into the visualization tool we’ll use Excel for this and then generate some pivot tables and pivot charts we will only do this for one file which is 2020 right now what’s the surprising part is this particular file will also get data continuously for the next year 2021 for the next year 2022 for the next year 2023 and the current year 2024 right when we just copy paste or cut copy paste these files into this particular folder the same transformation steps the same data cleaning steps and the same data loading steps and also including the analysis will be done for you in the Excel pivot table right the process has been just automated right so you can see that happening for that let’s open a new Excel file so I have the Excel folder right over here so a new blank Excel workbook will be opened and if you go to data here you have get data as we said it can be a file it can be an Excel file XML Json PDF it can be from cloud it can be from a SQL database any any other source if it’s data you can work from it so we will give a file here right so just uh uh okay so so we will give a file here so select file and go to the last option which reads us from folder right so go to downloads and select that AIT data yeah go to downloads and select that a this data remember this particular folder should have only one file according to us which is 2020 data the first year data right select the folder and open and this should include the data of only one file which is AUST 2020 right so if you have have multiple files you can either combine transform and load if you have just one file you can just go with the transformation option over here or you can load if you are comfortable with what you have so I feel there are some minor changes which I would like to make to my data so I would go with the transform option now the power query window will shortly open now with the power query window opened you can select the orders 2022 Excel folder and it will be loaded now let’s try to cck click on the content button over here which should load our data now you can go with the first file so here’s the preview as we expected we have the order ID order date delivery date Etc just press on okay so the data has been successfully loaded right now we have the First Column as Source name which we will not be needing anymore and you can just remove that order ID is fine order date is fine is it no right so you can just select both the columns and change the data type right so here you have the option of change the data type where you will change the data type from whole number to date so that you get a date right so you have that the First Transformation step changing data type has been recorded here so this is like uh we uh Excel vbx or macro which will save all the steps and will apply uh to all the uh upcoming files right you have to change the type of delivery date as well so we will be finding uh the how many days it took to deliver or if if the delivery got delayed right so we can test that and here if you see uh yes we requested customer ID and customer name so we got customer ID along with the name so we don’t want that we want that in two separate columns we can do that so you can just select the entire column go to the split column option over here select by D limiter so I think it all automatically recognize that for us but there is one loophole we have a space Del limiter space option so just press okay and they should be helping us with the customer ID so let’s rename the customer ID to just customer ID and this to uh customer name space name right and that’s done here we have product which is fine quantity is fine traffic is fine leads is fine orders and here comes the money part so we have Revenue cost to company so we don’t know the profit yet right so to calculate the profit what you need to do is substract revenue from cost to the company and automatically you will get your profit so in some instances you might get a negative value don’t worry about that because uh that can be a discount offer or some 50% off 75% off that you regularly sell out your products which are selling in less num so that you can just you know uh move out the products from the store so that can be done and order status so we have order status as delivered in transit not delivered yet or order cancelled right so we might have to rule out some uh status here right right so we are calculating profit here so we don’t want the profit for cancelled orders or not delivered okay maybe okay okay not delivered orders uh this might be considered as the cost of company and revenue because uh it takes some delay to refund the amount to the customers so we can just add a filter to this particular column and and uh we will unselect not delivered products and Order cancel products so not delivered happens when the vendor is not available to deliver the product to the delivery agencies so in that case uh the order gets automatically cancelled and uh the order will be the money will be refunded to the customer so we will rule out these now the data is clean now uh as we thought to calculate the profit select these two columns first Revenue then cost a company because we are subtracting cost to company from revenue right so we will go to add to column and standard and here we have some options so we will subtract that so we will get the profit here so now you can name it as profit done so we just move that column to the money part of this particular data set so profit cost to company and revenue are the money parts so we will change data typ from just values to currency right so then that should be converted into the dollars format now the first one we don’t need this Source name so you can just rule out this you can just uh remove this column and there you go now you have a clean data you have the order ID order date delivery date yeah we forgot uh the another part of this particular data set transformation where we wanted to find out the number of dat it took to deliver the product okay I think we changed order ID so we don’t want this so the first okay let’s change this to the to the whole number as it was okay so order ID fine now we wanted to find out the number of days it took to deliver the product so we will select the delivery date first and then the order date second go to add column and uh we will select subtract in days and we should have that now let’s change the uh substract subtraction name to days to Der and you can move this column right select the whole column and hold it drag it and you can move this column along with the dates part of this data set where we have the ordered date and delivered date right right over here and you will find how many days it took to deliver a certain product now um let’s recheck once order ID order date delivery date dat to deliver customer ID customer name what product did he or she ordered and what quantity and uh the traffic which we received to the website the traffic which we received to the website the number of leads we got out of those traffic numbers the number of orders we got out of those uh uh lease and what’s the revenue we got out of those orders and what was theost the company and what is the profit that we made out of that order right and we also have the order status remember we only considered remember we only considered two types right uh we considered in transit and delivered statuses only we ruled out cancelled orders and not delivered orders so that we have accurate results of our profit made and revenue made out of this right now I’m completely satisfied with my transformation and cleaning of my dat set now let’s go to home and here we have the option of close and load so we will go with that and this particular dat set should be loaded to the data visualization where we have selected Excel right uh so we have the ERS data for 20 20 correct okay let’s not add date here because after all we will have some more data for 2021 2 3 and four right so let it be the way it is and if you just check the filter here we have only one year and if you just check the filter here you will have only one year and if you just check the filter here you will have only one year 2020 and if you go to the delivery date which let’s say if you have an order placed on December 31st and it got delivered on January 5th or something so you’ll have a little bit of a part of 2021 data which is completely all right not to be worried about so the main important thing is audit date so we want the audit dates of 2020 so that is loaded here right now you can just select the whole data set over here and uh uh you can add or insert some pivot tables here right from the selected range and uh let’s go with the default option which creates a new sheet and everything in a new location right here you have it now let’s say you wanted to make out some insights from this uh data set let’s delete the other sheet which we don’t want and let’s name it as pivot now the products will be loaded to the columns and uh profit will be loaded to values so you have a data set over here sum of profit from beds chairs cupboards sofa tort so here you have the pivot table which has sum of profit bed chair cupboard soofa T and grand total sum of profits made from these these these and these now select this entire data and go to the sech option here go to the pivot chart and uh here automatically here you can see that it is you know suggesting or recommending us to go for the bar chart which is a lot more better to s okay and you have your PIV chart here right and see the grand total here 22 4811 4.15 now this is only for one years of data now what we will do is go back to the downloads folder select the remaining years which is 2021 2 3 and 4 add that to the file location so let’s quickly go to our folder here and go to downloads select the folders I’m going to cut them from here go to a data and paste them here right now our job is done if we go back to the data here go to home and data okay so now that we have added the data to our folder go to the a data and here select the data segment and just select refresh all so everything is refreshed now let’s check the audit date filter and we will have all the five years from 2020 to 1 2 3 and four and let’s press okay for that and if you go to the pivot table you can also see the changes here just press refresh all and there you go you have the updated data right over here now that saved a lot of time for you and now that time can be utilized to plan a strategy and focus on the lower parts of your business and improvise them and bring a big change to the revenue and with that we have come to an end of this session on how to use Microsoft powerquery so here you can see that I’ve got some sales data of various car manufacturing companies and I’ve also got some slices so using these slicers I can get the realtime information from the dashboard so I’ll select an E from the first slicer and I’ll go into the second slicer and I’ll select a company now I’ll select BMW and SUV type so we have the data of all the SUV types cars manufactured by BMW car manufacturing company in the year 2018 now let’s start and create something similar to this one so how did I create this mis report let’s go through this in a step-by-step way so on my screen you can see that we have created a completely new Excel workbook and we have some sample data with five rows and 20 columns so this particular data is is based on the manufacturer year and the car manufacturing company and the type of the car manufactured and number of cars manufactured in that particular year and also the price of that particular car now to create Mis reports we need to play around P tables so we also have tutorials on P tables and you can go through that in our simply learn YouTube channel and also we will try to link those in the description box below now let’s select the entire data and go into the insert ribbon and there you can see the first option that is pivot table so after selecting the pivot table option it will show this particular dialogue box where it will ask you for the range of cells you need to select and the next one is to create the P table in the same sheet or a new sheet so we’ll take a new sheet here now select okay and there you go you have the pivot table here so first you have an empty pivot table and on the right side you can see we have p table Fields now you can either drag these options into the rows and columns or you can also click it so now let’s drag and drop these first we will select here and car followed by that we will take type quantity and price to columns so there you go we have our pivot table now let’s select all the data in the pivot table table and again let’s get back to the insert option and here you can see an option called pivot chart select that and you will have various pivot chart options here now you can select any one which you can prefer or which you like or whichever you think can represent your data in a best way possible so now let’s go with column chart first let’s select okay and we have our column chart over here now this happens to be our first chart now let’s create a few more charts so to create few more charts we’ll go back to the same original data select again insert and pivot table now again a new worksheet so here we have another Mt P table in the similar way let’s drag the same data into Ros and columns there you go and again we will select all the data and create a pivot chart go to insert option pivot chat and now let’s try to select pie chat select okay and we have our pie chart over here now let’s try to to create one more same procedure select table select pivot table then create new worksheet for the new pivot table and now drag and drop the data now let’s go to insert again let’s select all the data go to insert option and pivot chart now let’s try to select the bar chart select okay and there we have our bar chart now remember the first chart we created that is in sheet two this one so we will try to move all the charts to this particular table or this particular worksheet so let’s get back to the sheet number three right click and you can see an option called pivot chart analyze so from here you can move the chart select the option of moving the chart and don’t select the first one select the object in option and select the sheet number to which you want to move this chart for now we want to move this chart to sheet number two select that press okay now we have the chart moved to the sheet number two and in the same way let’s go to the sheet number four and let’s move this particular chart to sheet number two there you go the chart got moved to sheet number two now you can insert the slices so we have selected the third sheet or third chart and inside the pivot chart analyze option we have an option of inserting the slicer now we will insert the slicers for ear car and type now select okay and you will have all the three slices onto your sheet now let’s rearrange them so I’m just organizing them in a neat way there you go and you can also customize your designs for all the charts so that they look a little more better now if you see if I make some changes only one chart will be interacting for this right so if I select Ferrari and year 2020 and option as Sports you can see only one chart is interacting to these particular slices why because all these slices are connected to just one chart here so you need to make sure that all these slices are connected to all the three charts what we created so let’s erase all the filters and select one slicer at a time right click and you will see an option of report connections select that and you see that the connections are only connected to the pivot table 3 so select the remaining pivot tables as well select okay and in the same way select the second slicer select the report connection option connect pivot table 1 and two and select okay and in the same way the third one so now all the slices are connected to all the pivot charts now if you make some changes here they will respond automatically in the same way now let’s select the for company the year 2019 and hatchback option and you have all the cars which were made in the year 2019 from Ford Company in the segment of hashack so this was the sample data set that we worked on now you can also work on some complex data set such as this Superstore data set now one thing you need to make sure that when you’re working with Exel Mis they need to make sure that all the macros are disabled on your data set and you convert the entire data into the regular table format of excel right now Excel is considering this particular data as a database so for that you need to select all the datar Press contrl T and then convert the entire data in the form of tabular data and also don’t forget to click this icon which reads my table has headers so the first row is the headers now let’s select okay and your entire data will be converted into tabular form now let’s reset this to the normal type which has the clear color not all the fancy colors now the entire data is converted from the dbms to the tulet form and now you can start implementing the Mis reports and PIV tables and uh so why exactly do we need to do time series analysis typically we would like to predict something in the future and uh it could be stock prices it could be the sales or um anything that needs to be predicted into the future that is when we use time series analysis so it is um as the name suggests it is forecasting and typically when we say predict it need not be into the future in machine learning in data analysis when we talk about predicting we are not necessarily talking about the future but in Time series analysis we typically predict the future so we have some past data and we want to predict the future that is when we perform time series analysis so what are some of the examples uh it could be daily stock price the shares as we talk about or it could be the interest rates weekly interest rates or sales figures of a company so these are some of the examples where we use time series data we have historical data which is dependent on time and then based on that we create a model to predict the future so what exactly is uh time series so time series data has time as one of the components as the name suggests so in this example let’s say this is the stock price data and uh one of the components so there are two columns here column B is the price and column A is basically the time information in this case the time is a day so the primarily the closing price of a particular stock has been recorded on a daily basis so this is a Time series data and the time interval is obviously ly a day time series or time intervals can be daily weekly hourly or even sometimes there is something like a sensor data it could be every few milliseconds or micros seconds as well so the size of the time intervals can vary but they are fixed so if I’m saying that the it is daily data then the interval is fixed as daily if I’m saying this data is an hourly data then it is the data is captured every hour and so on so the time intervals are fixed the interval itself you can uh decide based on what kind of data we are capturing so this is a graphical representation the previous one here we saw the table representation and this is how to plot the data so on the Y AIS is let’s say the price or the the stock price and x-axis is the time so against time if you plot it this is how a Time series graph would look so as the name suggest what is time series data time series data is basically a sequence of data that is recorded over a specific intervals of time and based on the past Valu so if we want to do an analysis of Time series past data we try to forecast a future and uh again as the name suggests it is time series data which means that it is time dependent so time is one of the components of this data time series data consists of primarily four components one is the trend then we have the seasonality then cyclicity and then last but not least regularity or the random component sometimes is also referred to as a random component so let’s see what each of these components are so what is Trend trend is overall change or the pattern of the data which means that the data maybe let me just uh pull up the pen and uh show you so let’s say you have a data set somewhat like this a Time series data set somewhat like this all right so what is the overall trend there is an overall trend which is upward Trend as we call it here right so it is not like it is continuously increasing there are times when it is dipping then there are times when it is increasing then it is decreasing and so on but overall over a period of time from the time we start recording to the time we end there is a trend right there is an upward Trend in this case so the trend need not always be upwards there could be a downward Trend as well so for example here there is a downward Trend right so this is basically what is a trend overall whether the data is increasing or decreasing all right then we have the next component which is seasonality what is seasonality seasonality as the name suggests once again changes over a period of time and periodic changes right so there is a certain pattern um let’s take the sales of warm clothes for example so if we not it along the months so let’s say January February March April May June July and then let’s say it goes up to December okay so this is our December D I will just mark it as D and then you again have Jan FB March and then you get another December okay and just for Simplicity let’s mark this as December as the end of the year and then one more December okay so what will happen when if you’re talking about warm clo clothes what happens the sales of warm clothes will increase probably around December when it is cold and then they will come down and then again around December again they will increase and then the sales will come down and then there will be again an increase and then they will come down and then again an increase and then they will come down let’s say this is the sales pattern so you see here there is a trend as well there is an upward Trend right the sales are increasing over let’s say these are multiple years this is for year 1 this is for year two this is for year three and so on so for multiple years overall the trend there is an upward Trend the sales are increasing but it is not a continuous increase right so there is a certain pattern so what is happening what is the pattern every December the sales are increasing or they are peing for that particular year right then there is a new year again when December approaches the sales are increasing again when December approaches the sales are increasing and so on and so forth so this is known as seasonality so there is a certain fluctuation which is uh which is periodic in nature so this is known as seasonality then cyclicity what is cyclicity now cyclicity is somewhat similar to seasonality but here the duration between two cycles is much longer so seasonality typically is referred to as an annual kind of a sequence like for example we saw here so it is pretty much like every year in the month of December the sales are increasing however cyclicity what happens is first of all the duration is pretty much not fixed and the duration or the Gap length of time between two cycles can be much longer so recession is an example so we had let’s say recession in 2001 or 2002 perhaps and then we had one in 2008 and then we had probably in 2012 and so on and so forth so it is not like every year this happens probably so there is usually when we say recession there is a slump and then it recovers and then there is a slump and then it recovers and probably there is another bigger slump and so on right so you see here this is similar to seasonality but first of all this length is much more than a year right that is number one and it is not fixed as well it is not like every four years or every six years that duration is not fixed so the the duration can vary and at the same time the gap between two cycles is much longer compared to seasonality all right so then what is irregularity irregularity is like the random component of the time series data so there is like you have part which is the trend which tells whether the overall it is increasing or decreasing then you have cyclicity and seasonality which is like kind of a specific pattern right uh then there is a cyclicity which is again a pattern but at much longer intervals plus there is a random component so which is not really which cannot be accounted for very easily right so there will be a random component which can be really random as the name suggests right so that is the irregularity component so these are the various components of Time series data yes there are conditions where we cannot use time series analysis right so is it can we do time series analysis with any kind of data no not really so so what are the situations where we are uh we cannot do time series analysis so there will be some data which is collected over a period of time but it’s really not changing so it will not really not make sense to perform any time series analysis over it right for example like this one so if we take X as the time and Y as the value of whatever the output we talking about and if the Y value is constant there is really no analysis that you can do uh leave a time series analysis right so that is one another possibility is yes there is a change but it is changing as per a very fixed function like a sine wave or a c wave again time series analysis will not make sense in this kind of a situation because there is a definite pattern here there is a definite function that the data is following so it will not make sense to do a Time series analysis now before performing any time series analysis uh the data has to be stationary and uh typically time series data is not stationary so in which case you need to make the data stationary before we apply any models like ARA model or any of these right so what exactly is stationary data and what is meant by stationary data let us take a look first of all what is non-stationary data time series data if you recall from one of my earlier slides we said that time series data has the following four components the trend seasonality cyc and random random component or irregularity right so if these components are present in Time series data it is non-stationary which means that typically these components will be present therefore most of the time A Time series data that is collected raw data is non-stationary data so it has to be changed to stationary Data before we apply any of these algorithms all right so a nonstationary Time series data would look like this which means like for example here there is an upward Trend the seasonality component is there and also the random component and so on so if the data is not stationary then the time series forecasting will be affected so you cannot really perform a Time series forecasting on a non-stationary data so how do we differentiate between a stationary and a non-stationary Time series data typically or technically one is of course you can do it visually in non-stationary data the the data will be more flattish the seasonality will of course be there but the trend will not be there so the data May if we plot that it may appear somewhat like this right it’s a horizontal line along the horizontal line you will see compared to the original data which was there was an upward Trend so it was changing somewhat like this right so this is non-stationary data and this is how a stationary data would look visually what does this mean technically this means that stationarity of the data depends on a few things what the mean the variance and the co-variance so these are the three components on which the stationarity of the data depends so let’s take a look at what each of these are for stationary data the mean should not be a function of time which means that the mean should pretty much remain constant over a period of time right so there is there shouldn’t be any change uh so this is how the stationary data Outlook and this is how a non-stationary data Outlook I’ve shown in the previous slide as well so here the mean is increasing that means there is an upward Trend okay so that is one part of it and then the variance of the series should not be also a function of time so the variance also should be pretty much common or should be constant rather uh so this is a if we visually we take a look this is how time series stationary data would look where the variance is not changing here the variance is changing therefore this is nonstationary and we cannot apply Time series is forecasting on this kind of data similarly the co-variance which is basically of the ath term and the i+ MTH term should not be a function of time as well so co-variance is nothing but not only the variance at the ith term but the relation between the variance of the ith term and the I plus MTH or the I plus n term so as again once again visually this is how it would look if the co-variance is also changing with respect to time so these are the all three components should be pretty much constant and that is when you have stationary data and in order to perform time series analysis the data should be stationary okay so let’s take a look at uh the concept of moving average or the method of moving average and let’s see how it works we’ll do simple calculations so let’s say this is our sample data we have the data for three months January February March the sales in hundreds of in thousands rather not hundreds thousands of dollars is given here and uh now we want to find the moving average so how do we find the moving average we call it as moving average three so moving average three is nothing but you take three of the values or the readings add them up and uh divide by three basically the way we take a mean or average of the three values so that is as simple as that so that’s the average first of all so what is moving average moving average is if you now have a series of data you keep taking the three values the next next three values and then you take the average of that and then the next three values and so on and so forth so that is how you take the moving average so let’s take a little more detailed example of car sales so this is how we have the car sales data for the entire year let’s say so rather for four years so year one we have for each quarter quarter 1 2 3 4 and then year two quarter 1 2 3 4 and so on and so forth so this is how we have sales data of a particular car let’s say or a showroom and uh we want to forecast for year five so we have the data for four years we now want to forecast for the fifth year let’s see how it works first of all if we plot the data as it is uh taken the raw data this is how it would look and uh what do you think it is is it stationary no right because there is a trend upward Trend so this is not a stationary data so we um we need to later we will see how to make it stationary but to start with just an example we will not worry about it for now if we will just go ahead and uh manually do the forecasting using what is known as moving average method okay so we are not applying any algorithm or anything like that in the next video we will see how to apply an algorithm how to make it stationary and so on all right so um here we see that all the three or four components that we talked about um are there there is a trend there is a seasonality and then of course there is some random component as well cyclicity may not be it is possible that cyclicity is not applicable in all the situations for sales especially there may not be or unless you’re taking a sales for maybe 20 30 years cyclicity may not come into play so we will just consider uh primarily the trend seasonality and irregularity right so Random it is also known as random irregularity right so we were calling the random or irregularity component so these are the three main components typically in this case we will talk about so this is the component and um we will see how to do these uh calculations so let’s take a look red draw the table including the time code we will add another column which is the time code and uh this the column and we’ll just number it like 1 2 3 4 up to 16 the rest of the data Remains the Same okay so we will do the calculations now now let us do the moving average calculations um or ma4 as we call it for each year so we take all the four quarters and we take an average of that so if we add up these four values and divide by four you get the moving average of 3.4 so we start by putting the value here so that will be for the third quarter let’s say 1 2 3 the third quarter then we will go on to the next one so we take the next four values as you see here and take the average of that which is the moving average for the next quarter and so on and so forth now if we just do the moving average uh it is not centered so what we do is we basically add one more column and we calculate the centered moving average as shown here so here what we do is we take the average of two values and then just adding these values here so for example the first value for the third quarter is actually the average of the third and the fourth quarter so we have 3.5 now it gets centered so similarly the next value would be 3.6 + 3.9 ided 2 so which is 3.7 and so on and so forth okay so that is the centered moving average this is done primarily to smooth the data so that there are not too many rough edges so that is what we do here so if we visualize this data now uh this is how it looks right so if we take the centered moving average as you can see there is a gradual increase if this was not the case if we had not centered it the changes would have been much sharper so that is the basically the smoothening that we are talking about now let’s go and or do the forecast for the fifth year so in order to do the forecast what we will do is we will take the centered moving average as our Baseline and then start doing a few more calculations that are required in order to come up with the prediction so what we are going to do is we are going to use this multiplicity or M multiplicative model in this case and this is how it it looks so we take the product of seasonality and uh the trend and the irregularity components and we just multiply that and in order to get that this product of these two We have basically the actual value divided by CMA YT value divided by CMA will give you the predicted value on YT is equal to the product of all three components therefore St into Y is equal to YT by CMA so this is like this is equal to YT right so therefore if we want St into YT the product of seasonality and irregularity is equal to YT by CMA so that is how we will work it out I also have an Excel sheet of the actual data so let me just pull that up all right so this is how the data looks in Excel as you can see here year 1 quarter 1 2 3 4 year 2 quarter 1 2 3 4 and so on and this is the sales data and then this is the moving average as I mentioned this is how we calculate and this is the centered moving average so this is the primary component that we will start working with and then we will calculate since we want the product of St into ITT that is equal to YT by CMA so if you see these values are nothing but the YT value divided by CMA so in this case it is 4 by 3.5 which is 1.14 similarly 4.5 by 3.7 1.22 and so on and so forth so we take we have the product St into it and uh then the next step is to calculate the average of respective quarters so that is what we are doing here average of respective quarters and then we need to calculate the deseasonalized values so in order to get deseasonalized value we need to divide YT by St that was calculated so for example here it is 2.8 by .9 so we got the decisional value here and uh then we get the trend and then we get the predicted values so in order to get the predicted value which is basically we predict the values for known values as well like for example year one quarter one we know the value but now that we have our model we predict ourselves and see how close it is so we predicted as 2.89 whereas the actual value is 2.8 then we have 2.59 the actual value is 2.1 and so on just to see see how our model works and then continue that into the fifth year because for fifth year we don’t have a reference value okay and if we plot this we will come to know how well our calculations are how well our manual model in this case we did not really use a model but we did on our own manually so it will tell us the trend so for example the predicted value is this gray color here and you can see that it is actually pretty much following the actual value which is the blue color color right and the gray color is the predicted value so the wherever we know the values up to year four we can see that our predicted values are following or pretty much very close to the actual values and then from here onwards when the year five starts the blue color line is not there because we don’t have the actual values only the predicted values so we can see that since it was following the trend pretty much for the last four years we can safely assume that it has understood the pattern and it is predicting correctly for the next one year the next four quarters right so that is what we are doing here so these four quarters we did not have actual data but we have the predicted values so let’s go back and see how this is working in this using the slides so this is we already saw this part and um I think it was easier to see in the Excel sheet so we calculated the St it the product of St and it using the formula like here y by YT by CMA we got that and then we got ST which is basically YT so this is average of the first quarters for all the four years and uh similarly this is the average of the second quarter for all the four years and so on so these values are repeating there are they are calculated only once they get repeated as you can see here and uh then we get the deseasonalized data and that is basically YT by St so we calculated St here and we have YT so y YT by St will give you the deseasonalized data and uh we have got rid of the seasonal and The Irregular components so far now what we are left with is the trend and uh before we start the time series forecasting or time series analysis as I mentioned earlier we need to completely get rid of the non-stationary components so we are still left with the trend component so now let us also remove the trend component in order to do that we have to find the or we have to calculate the intercept and slope of the data because that is required to calculate the trend and uh how are we going to do that we will actually use um what is known as a regression tool or Analytics tool that is available in Excel so you remember we have our data in ex Excel so let me take you to the Excel and uh here we need to calculate the intercept and the slope in order to do that we have to use the regression mechanism and in order to use the regression mechanism we have to use the Analytics tool that comes with Excel so how do you activate this tool so this is how you would need to activate the Tool uh from Excel you need to go to options and uh uh in options there will be addin and in addin you will have um analysis tool pack and you select this and um you just say go it will open up a box like this you say analysis tool pack and you say Okay And now when you come back in to the regular view of excel in the data tab you will see data analysis activated so you need to go to file options and addins and then analysis tool pack typically since I’ve already added it it is coming at the top but it would come under inactive application addin so when you’re doing it for the first time so don’t use VBA you just say analysis tool pack there are two options one with VBA like this one and one without VBA so just use the one without VBA and then instead of just saying okay just take care that you click on this go and not just okay so you say go then it will give you these options only then you select just the analysis tool pack and then you say okay all right so and then when you come back to the main view you click on data okay so this is your normal home view perhaps so you need to come to data and here is where you will see data analysis available to you and then if you click on that there are a bunch of possibilities what kind of data anal analysis you want to do if there are options are given right now we just want to do regression because we want to find the slope and The Intercept so select regression and you say okay and you will get these options for input y range and input X range input y range is the value YT so you just select this and you can select up to here and press enter and input X range you can for now you start with uh the baseline or you can also start with the D seasoned values so you can just click on these and say okay I have already calculated it so these are the intercept and the coefficients that we are getting for these values and we will actually use that to calculate our Trend here right so which is in the J colum so trend is equal to intercept plus slope into the time code so The Intercept is uh out here as we can see in our slide as well so if you see here this is our intercept and the lower value is the slope we have calculated here and it’s shown in the slides as well so intercept the formula is shown here so our trend is equal to intercept plus slope into time code time code is nothing but this one t column A 1 2 3 4 okay so that’s how you calculate the trend and that’s how you use the data analysis tool from Excel using these two we calculate the predicted values and using this formula which is basically trend is equal to intercept plus slope into time code and then we can go and plot it see how it is looking and therefore so we see here that the predicted values are pretty close to the actual values and um therefore we can safely assume that our calculations which are like our manual model is working and hence we we go ahead and predict for the fifth year so till four years we know the actual value as well so we can compare how our model is performing and for the fifth year we don’t have reference values so we can use our equations to calculate the values or predict the values for the fifth year and we can go ahead and safely calculate those values and when we plot for the fifth year as well the predicted values we see that they are pretty much they captured the pattern and we can safely assume that the predictions are fairly accurate as we can also see from the graph in the Excel sheet that we have already seen okay so let’s go and plot it so this is how the plot looks this is the CMA or the centered moving average the green color and then the blue color is the actual data red color is the predicted value predicted by our handcrafted model okay so remember we did not use any regular for forecasting model or any tool we have done this manually and uh the actual tool will be used in the next video this is just to give you an idea about how behind the scenes or under the hood how fusting Works a Time series analysis how it is performed okay so it looks like it has captured the trend properly so up to here is the known reference we have reference and from here onwards it’s purely predicted and uh as I mentioned earlier we can safely assume that the values are accurate and predicted properly for the fifth year so let’s go ahead and Implement a Time series forecast in r first of all we will be using the arima model to do the forecast of uh this time series data so let us try to understand what is arima model so arima is actually an acronym it stands for autor regressive integrated moving average so that is what is ARA model and it is specified by three parameters which is p d and q p stands for autor regressive so let me just mark this so there are three components here Auto regressive integrated moving average okay so these three parameters correspond to those three components so the P stands for auto regressive D for integrated and Q for moving average so let us see what exactly this is so these three factors are p is the number of Auto regressive terms or ar we will see that in a little bit and D is how many levels of differences that we need to do or differentiation we need to do and Q is the number of lagged forecast errors so we’ll see what exactly each of these are so a is the number of autor regressive terms and which is basically denoted by the p and then we have D which is for the number of times it has to be differentiated and then we have q which is for the moving average so what exactly AR terms so in terms of the regression model Auto regressive components refer to the prior values of the current value well what we mean by that is here when we talk about time series data focus on the fact that there is regression so what exactly happens in regression we try to do something like if it a simple linear regression we do some equation like y = mx + C where there are actually there are two variables one is the dependent variable and then there is an independent variable let me just complete this equation as well MX plus C right so this is a normal regression curve or a simple regression curve now here we are talking about Auto regression or Auto regressive so Auto regressive as the name suggests is regression of itself so which means that here you have only one variable which is your maybe the cost of the flights or whatever it is right and the other variable is basically time dependent and therefore the value at any given time and that we will denote as YT for example so there is no X here there is only one variable and which is y and we say YT which is basically the predicted value at a time interval T for example is dependent on the previous Valu so for example there may be A1 and then YT minus1 and then there will be like plus A2 and right plus A2 and YT minus 2 and uh all right and then plus A3 into y tus 3 all right so basically here what we are saying is there’s only one variable here but there is a regression component so we are doing a regression on itself so that’s how the term Auto regression comes into play so only thing is that it is dependent on the pre previous time values so there is a lag let’s say this is the first lag second lag third lag and so on so the current value which is YT is dependent on the previous time lag values so that is what is auto regression component so this is what is shown here for example in this case instead of Y we are calling it as X so that’s the same and this is represented by some equation of that sort depending on how many lags we take so that is the AR component and the term p is BAS basically determines how many lags we are considering so that’s the term e for now what is d d is the degree of differencing so here differencing is like to for the nonseasonal differences right so for example if you take the values like this which are given for five 4 six and so on and so forth if you take the differencing of one after another like for example 5 – 4 or 4 – 5 the next value with the previous value so 4 – 5 so this is known as the first order differencing so the result is min -1 similarly 6 – 4 is 2 7 – 6 is 1 so this is first order differencing and uh here we call it as D is equal to 1 okay and same way we can have second order third order and so on then the last one is q q is the actually by we call it moving average but in reality it is actually the error of the model so we also sometimes represent as ET all right so now ARA model works on the assumption that the data is stationary which means that the trend and seasonality of the data has been removed that is correct okay so this we have discussed in the first part how what exactly is stationary data and how do we remove the non-stationary part of it now in order to test whether the data is stationary or not there are two important components that are considered one is the autocorrelation function and other is the partial autocorrelation function so this is referred to as ACF and pacf all right so what is autocorrelation and what is the definition correlation is basically the similarity between values of a same variable across observations as the name suggest now how do we actually find the aut correlation function the value right so this is basically done by plotting and autocorrelation function also tells you how correlated points are with each other based on how many time steps they are separated by and so on that is basically the time lag that we were talking about and it is also used to determine how past and future data points are related and the value of the autocorrelation function can vary from minus1 to 1 so if we plot this is how it would look autocorrelation function would look somewhat like this and there is actually a readily available function in R so we will see that and you can use that to plot your a corelation function okay so that is ACF and we will see that in our R studio in a little bit and similarly you have partial autocorrelation function so partial autocorrelation function is the degree of association between two variables while adjusting the effect of one or more additional variables so this again can be measured and it can also be plotted and its value once again can go from minus1 to one and it gives the partial correlation of Time series with its own lagged values so lag again we have discussed in the previous uh couple of slides this is how PF plot would look in our studio we will see that as well and once we get into the r studio and with that let’s get into our studio and take a look at our use case before we go into the code let’s just quickly understand what exactly is the objective of this use case so we are going to predict some values or forecast some values and we have the data of the airline ticket sales of the previous years and now we will try to find the or predict the forecast the values for the future years all right so we will basically identify the time series components like Trend seasonality and uh random Behavior we will actually visualize this in our studio and then we will actually forecast the values based on the past values or history data historical data so these are the steps that we follow we will see in our studio in a little bit just quickly let’s go through what are the steps we load the data and it is a Time ser data if we try to find out what class it belongs to the data is actually air passengers data that is already comes preloaded with the our studio so we will be using that and we can take a look at the data and then what is the starting point what is the end point so these are all functions that are really available we’ll be using and then what is the frequency it’s basically frequency is 12 which is like yearly data right so every month the data has been collected so for each year it is 12 and then we can check for many missing values if there are any and then we can take a look at the summary of the data this is what we do in exploratory data analysis and then we can plot the data visualiz the data how it is looking and we will see how the data has some Trend seasonality and so on and so forth all right then we can take a look at the cycle of the data using the cycle function and we can see that it is every month that’s the cycle end of every 12 months a new cycle begins so each month of the year is uh the data is available then we can do box plots to see for each month how the data is varying over the various 10 or 12 years that we will be looking at this data and uh from exploratory data analysis we can identify that there is a trend there is a seasonality component and how the seasonality component varies also we can see from the box plots and we can decompose the data we can use the decompose function rather to see the various components like the C ality Trend and the irregularity part okay so we will see all of this in our studio this is how they will look this is the Once you decompose and this is how you will actually you can visualize the data this is the actual data and this is the trend as you can see it’s going upwards this is the seasonal component and this is your random or irregularity right so we call it irregularity or we can also call it random as you can see here yes so the data must have a constant variance and mean which means that it is stationary before we start any analysis time series analysis and without so basically yeah if it is stationary only then it is easy to model the data perform time series analysis so we can then go ahead and fit the model as uh we discussed earlier we’ll be using ARA model there are some techniques to find out what should be the parameters so we will see that when we go into our studio so the auto ARA function basically tells us what should be the parameters right so these parameters are the p d and q that we talked about that’s what is being shown here so if we use autoa it will basically take all possible values of this PDQ these parameters and it will find out what is the best value and then it will recommend so that is the advantage of using Auto ARA all right so like in uh this case it will tell us what if we use this parameter tra we set the parameter Trace is equal to true then it will basically tell us what is the value of this AIC which has to be minimum so the lower the value the better so for each of these combinations of p d and q it will give us the values here and then it will recommend to us which is the best model okay because whichever has the lowest value of this AIC it will recommend that as your best PDQ values so once we have that we can see that we will basically we can potentially get a model or the equation model is nothing but the equation and based on the parameters that we get and we can do some Diagnostics we can do some plotting to see how whether there is a plot for the residuals so which shows the stationarity and then we can also take a look at the ACF and PF we can plot the ACF and PF and then we can do some forecasting for the future year so in this case we have up to 1960 and then we can see how we can forecast for the next 10 years which is 1970 up to 1970 and once we have done this can we validate this model yes definitely we can validate this model and uh to validate the findings we use uh

    junkbox test and this is how you just call box. test and then you pass these parameters and you will get the values that will be returned which will tell us whether this how accurate this model is or accurate the predictions are so the values of P are quite insignificant in this case we will see that and that also indicates that our model is free of autocorrelation and that will basically be it so let’s go back and into our R studio and uh go through these steps in uh real time so we have to import this Library forecast package is not installed you to go here and install the forecast package okay so that’s the easy way to install rather than so click on this install I will not do it now because I’ve already installed so the first time that’s only one time then after that you just have to load it into memory and then keep going so we will load this data called air passengers so by calling this data method and if you see the the data a passengers is loaded here and if we check for the class it is a Time series data TS data so we can check for the dates we can also view the data in a little bit and start date is 19 49 and January and our end date is 1960 December and the frequency is 12 which is like collected monthly so that is the frequency which is uh 12 here and then we check if there are any um missing values there are no missing values and then we take a look at the summary of the data this is all exploratory data analysis and then if you just display the data this is how it looks and then we need to decompose this data so we will kind of store this in an object TS data and then use that to decompose and store the new values let me just clear this for now and uh if we decompose basically as we have seen in the slides decomposing is breaking it into the trend seasonality and The Irregular or random components then you can go ahead and plot it so when you plot it you can see here let me Zoom this this this is our original plot or observed value as it is known as then we have decomposed the three parts which is basically the trend as you can see there is a upward Trend then the seasonal component so this is some regularly occurring pattern and then there is a random value which is basically you cannot really give any equation or function or anything like that so that’s what this plotting has done and then you can actually plot them individually as well so these are the individual plots for the trend for the seasonal component and the Rand component all right so now let’s take a look at the original data and see how the trend is in a way so if we do this linear regression line it will show that the it is going upwards and we can also take a look at the cycle that are there which is nothing but we have a frequency of 12 right so the Cycles will display that it is January February to December and then back to January February and so on and so forth and if we do box plot for the monthly data you will see that for each of the months right and over the 10 years that the data that we have we will see that there is a certain pattern right this is also in a way to find the seasonality component so while January February sales are relatively low around July August the sales pick up so especially in July I think the sales are the highest and this seems to be happening pretty much every year right so this is every year in July there seems to be a peak in the sales and then it goes down and slightly higher in December and so on so that is Again part of our exploratory data analysis and once again let’s just plot the data now as I said in order to fit into an ARA model we need the values of PD and Q now one way of doing it is there are multiple ways actually of doing it the earlier method of doing it was you draw the autocorrelation function plot and then partial aut correlation function plot and then observe that and where does this change and then identify what should be the values of p and Q and so on now R really has a very beautiful method which we can use to avoid all that manual process that we used to do earlier so what R will do is there is a method called Auto ARA and if we just call this Auto ARA method and it will basically go and test the ARA model for all possible values of this parameters PDQ and then it will suggest to you what should be the best model and it will return that best model with the right values of PD and Q so you we as data scientists don’t have to do any manual you know trial and error kind of uh stuff okay so we got the model now and uh this is the model it it has PDQ values are 211 PDQ and this is the seasonal part of it so we can ignore it for now and so if we want to actually understand how this has returned these values 21 one as the best one there is another functionality or feature where we can use this Trace function or Trace parameter so if you pass to Auto ARA the trace parameter what it will do is it will show you how it is doing this calculation what is the value of the AIC basically ASC is what you know defines the accuracy of the model the lower the better okay so for each each combination of PDQ it will show us the value of AIC so let’s run it before instead of me talking so much let’s run this if we run auto ARA with Trace you see here there is a red mark here that means it is performing it’s executing this and here we see the display right so it starts with certain values of PDQ and then it finds that value is too high so it starts with again with some 0 1 1 0er and so on and so forth and ultimately it tells us okay this is our best model you see here it says this is our best model 211 let’s go back and see did we get the same one yes we got the same one when we ran without Trace as well right now why is 211 let us see where is 211 here is our 211 and if you compare the values you see that 1017 is pretty much the lowest value and therefore it is saying this is our best model all other values are higher so that that’s how you kind of um get your model and now that you have your model what you have to do you need to predict the values right so before that let us just do some test of these values so for that you install T Series again if you are doing it for the first time you would rather use this package and install and say T Series and install it and then you just use this Library function to load it into your memory all right so now that we got our model using Auto ARA let us go ahead and forecast and also test the model and also plot the ACF and psf remember we talked about this but we did not really use it we don’t have to use that but at least we will visualize it and uh for some of the stuff we may need this T Series Library so if you are doing this for the first time you may have to install it and my recommendation is don’t use it in the code you go here and install tseries and I will not do it now because I’ve already installed it but this is a preferred method and once you install it you just load it using this libraries function and then you can plot your residuals and this is how the residuals look and you can plot your ACF and psf okay so this is how your PF looks and this is how your ACF looks for now there is really nothing else we need to do with ACF and PF this just to visualize how that how it looks but as I mentioned earlier we were actually using these visualizations or these graphs to identify the values of p d and q and how that was done it’s uh out of scope of this video so we will leave it at that and uh then we will forecast for the next 10 years how do we forecast that so we call forecast and we pass the model and we pass what is the level of accuracy that you need which is 95% and for how many periods right so basically we want for 10 years which is like 10 into 12 time periods so that’s what we are doing here and now we can plot the forecast value so you see this is the original value up to I think 62 or whatever and then it goes up to 72 this blue color is the predicted value let’s go and zoom it up so that we can see it better so from here onwards we forus and you can see that it looks like our model has kind of learned the pattern and this pattern looks very similar to what we see in the actual data now how do we test our model so we can do what is known as a box test and we pass our model here residuals basically with different lags and from those values here the P values here we find that they are reasonably low the P values which means our model is fairly accurate we’ll be creating two dashboards using a sample sales data set so if you want to get the data and the dashboard file that we’ll be creating in this demo then please put your email IDs in the comment section of the video our team will share the files via email now let’s begin by understanding what is a dashboard in Excel a dashboard is a visual interface that provides an overview of key measures relevant to a particular objective with the help of charts and graphs dashboard reports allow managers to get a high level overview of the business and help them make quick decisions there are different types of dashboards such as strategic dashboards analytical dashboards and operational dashboards an advantage of dashboards is the quick detection of outliers and correlations with Comprehensive data visualization it is time-saving as compared to running multiple reports with this understanding let’s jump into our demo for creating our Dash boards we’ll be using a sample sales data set let me show you the data set first so here is the sales data set that we’ll be using for our demo so this data set was actually generated using a simulator today we are going to talk about descriptive statistics in Microsoft Excel and I will walk you through a stepbystep guide on how to perform this in Excel but before we dive into that let’s first take a quick look at what descriptive statistics really means descriptive statistics is is all about summarizing and describing the key features of a data set it helps you get a quick snapshot of your data by calculating things like mean median mode standard deviation range and more these are essential to understanding patterns and Trends in your data without having to look at every single value it’s the first step in analyzing data before jumping into more complex statistics now that we have a basic understanding of what descriptive statistics is let’s get into Excel and see how you can easily calculate these statistics using the buil-in tools I am in Ms Exel and I have a sample of data in my Excel to perform the descriptive Statistics over there is an amazing feature in Excel that many people are not aware about is a data analysis tool pack if you have not come across this word the data analysis tool pack in Excel is an addin in that provides Advanced Data analysis tools for statistical and Engineering calculations it simp fies the complex data analysis Tas by offering the buil-in features to perform various statistical test and data processing without needing to write formulas manually so we need to ensure first if we have activated the tool pack correctly so for that go to file then options and then addin inside addin at the bottom we have manage and where you make sure you have selected the Excel addin and then click on go and then in the check box make sure you have chosen the analysis tool pack option and then click on the okay button now when you see the topmost bar in the data bar has a data analysis button now we are all set to perform the descriptive statistics to perform this I will click on this data analysis button and from there I will choose the descriptive statistics option and select okay now in the input range here we need to enter the range of cells containing your data for that I will click on this upper arrow and highlight the cells containing my data and press enter the next option is grouped by depicts how your data is arranged in your sheet if it is arranged in the columns just like mine you can obviously choose columns and if it is arranged rowwise then go for rows if you have levels on the first row of your data you can take on the checkbox your levels in the first row I don’t have that so I will skip this then if you go for the output options this is where you want your descriptive statistics result to be returned the first option that is output range is where you can specify a particular cell in this sheet only where your result to be returned second option is new worksheet that will answer your your descriptive statistics in a completely new worksheet the third option is new workbook we’ll enter the result all together in a different Excel file I want to enter my data in the same shet here so I’ll go for output range and I’ll choose somewhere here and I will press enter underneath I have various options available I will take all of them and don’t worry I will explain each in more detail so go for okay and here you go you have got the whole descriptive Statistics over here mean standard deviation and more now moving on the first value we got is the mean basically it is the average of all the data values the mean is actually the sum of all the data points divided by the number of the data points available in the sheet it tells you the central value of the data you can separately calculate it in Excel using the function average now moving on we have the second one which is standard error it is nothing but the measure of the variability of sample means in a sampling distribution of means if you want to calculate the standard error separately in Excel with the formula standard deviation divided by the square root of count now moving on we have median in short it is the middle value when data is sorted if I elaborate it then the median is the middle value in a data set when it’s ordered from smallest to the largest if the number of data points is odd is a middle number if it is even it is the average of the two middle numbers separately you can perform this in Excel by using the function median now we have mode it is basically the most frequent value rather it refers to the value that appears most frequently in a data set we can perform it separately in Excel using the function mode next we have a standard deviation the standard deviation measures the spread or dispersion of your data a higher standard deviation means the data points are spread out widely while a lower standard deviation means they’re close to the mean you can perform this by using the function stdv now coming to sample variance variance is the square of the standard deviation if it gives an idea of how data points spread out from the but in squar units to perform this you need to use a v function now we are going to kosis kosis measures the tailess of the data distribution High kosis means more outl of the heavy Tails while low kosis means the data has fewer extreme values describing the higher kosis and the lower kosis next we have skewness skewness measures the asymmetry of the data a positive SK skew means the data is skewed to the right more lower values while a negative skew means the skewed to the left that is more higher values now moving on we have range range is the difference between the maximum and the minimum values in your data set the maximum value we have over here is 9.23 and the minimum value we have is 11.67 so the difference is 78.5 6 to find out the minimum value in the Excel sheet use the function minimum to find out the maximum value in the Excel sheet use the function Max then coming to sum as you all know sum is the total addition of all the values in your sheet we can calculate sum separately by using the function sum then we have count this is the total number of values in the sheet in my case it’s 20 it will vary in everyone else then we have the largest the first written in the bracket denotes the first largest number that is 9.23 if the number would have been two it will denote the second largest number same goes for the smallest the smallest number in my case is 11.67 last but not the least we have confidence level this is the number we need to add and subtract from the mean the addition of the CL and mean is greater than 95% of the CL the subtraction of Cl from mean is lower than than 95% of Cl welcome to Simply learn in this video we are focusing on chat GPT and its groundbreaking role in data analytics in a world where data is the new gold understanding and analyzing this wealth of information is crucial but as the volume of data grows exponentially traditional analysis methods are being pushed to their limits enter CH GPT a revolutionary AI developed by openi that’s transforming how we approach data analytics but what makes chat jbt stand out in the crowded field of data analytics tools how can it not only streamline complex processes but also uncover insights that were previously hidden in plain sight stay tuned as we unrel the capabilities of chat GPT share real world applications and show you how it’s being used by analysis and businesses alike to make smarter datadriven decisions and whether you are a data scientist a business professional or just a tech Enthusiast curious about the future of AI in data analytics this we video is for you so let’s dive in and discover how CH GPT is not just revolutionizing data analytics but also how we understand and interact with the vast Universe of data and don’t forget to hit that like button subscribe and ring the bell to stay updated on all our future Explorations into the fascinating world of technology now let’s get started and unlock the potential of chat GPD for data analytics together so guys an Excel data analyst always looks for ways to improve their efficiency and gain deeper insight sites from the data that’s where chat jbt comes in handy so chat jbt is an AI powered language model that can assist you in various task including Excel based data analysis and for that data analysis let’s dive into some demos to see how it works so guys as you can see this is the chat jpt 4 and you could see this has the support of D browsing and Analysis and usage limits May apply like we have the version that has some usage limits and for the day data I have downloaded the data from the kar I will provide you the link in the description if you want to download the same data you can and I have that in my downloads so this is the data that we’ll be using so the first thing we’ll be doing is data cleaning and preparation so firstly let’s explore data cleaning and preparation before diving into analysis it’s essential to ensure that data is clean and ready for processing so for that we will write a prompt in chat JB and ask him to clean the data chat JP has the feature to insert the files provide the Excel file to the chat jpt and we’ll ask him that can you help me identify and remove duplicates in this Excel spreadsheet so let’s ask him can you help me identify and remove duplicates duplicates from this Excel spreadsheet from the provided Excel spreadsheet you could see that J has started generating the response and he is telling us that he can definitely help us with that and he is analyzing that so you could see that J has provided us the response so let’s first go to our spreadsheet so this is the spreadsheet order number quantity ordered price of each quantity and Order lines so you could see that first J GPT has encountered an error and that is with the character and coding Mis patch and then he has loaded the spreadsheet successfully with the second attempt and you could see that he has provided a like the smaller or the briefer version of the spreadsheet that is the data set consist of sales data with column such as order number quantity order price each order line sales order date status so he asking us could you specify how you would like to identify duplicates for example should we consider a row a duplicate if all its values match another row exactly or are there specifically columns we should focus on to determine duplicates so we should or here we will be focusing on the column section so here we want to identify column section so is asking which columns they should identify duplicates so we want the duplicates in order number so the CH G has started analyzing this spreadsheet so you could see that CH gpts address the response that they have identified a total of 287 rows in the spreadsheet that have duplicates based on the order number column so this indicates that multiple rows share the same order number suggesting there might be different items or transactions within the same order would you like to proceed by removing these duplicates while keeping either the first for last occurence of each order number or do you have another approach in mind for handling these duplicates so before identifying the duplicates what we’ll do is we’ll first get rid of the extra spaces so we’ll ask chat jbt that first find the Extra Spaces in the spreadsheet and help us remove move them so you could see that it has started analyzing it so he is asking us that would you like to proceed with any further cleaning or data manipulation yes we want to proceed further with data cleaning only and now we will treat all the blank cells if there are any now treat all the blank cells if there are any so the spreadsheet contains blank cells in the following columns with the respective counts of blank entries that is address line two has 2521 blank sales than the state so depending on your preference or the requirements of data analysis we can treat these blank cells in several ways such as filling them with the placeholder value that is not provided using a specific value that makes sense for data that is example a default state or TY but potentially leaving them as if blanks carry meaningful information in the context of your data so what we’ll do we will provide a placeholder provide a placeholder instead that is not provided or not applicable okay so you would see in that has responded that it has replaced the plank cells in the columns that is address line to State postal code and territory with the placeholder node provided so you could see there’s a verification sample showing the updates so this is how you could clean the data and now with chat jpt you can quickly identify and eliminate duplicates in your Excel spreadsheet using buil-in functions and formulas saving your time and effort so that would be done by the chat GPT and you can download the Excel file from the chat GPT only after that you can just ask him that provide me the output or the resulted spreadsheet so if we talk about now data analysis and visualization let’s delve into that so Excel offers powerful tools for analyzing and visualizing data and chat GPT can help you make the most of them so for this also we’ll write a prompt to the chat gy and we’ll write for example we’ll write please create a pivot table and a corresponding chart to analyze sales performance by region so let’s see like this prompt what output does chat Jey show to us so let’s ask him please create a PR table and a corresponding chart to analyze sales performance by region so you would see that jut has started analyzing the data and the spreadsheet and you could see that the pivot table below shows the sales performance per region or the territory so this table indicates that the EMA region has the highest Sal performance followed by regions where the territory was not provided than apek and finally Japan so now let’s create a chart to visually represent this sales per performance by region you could see that it has provided a chart that is a bar chart illustrating the sales performance by teritory so you could see the guys that it has provided us the analysis and J jpd can guide you through the process of creating pivot tables charts and other visualizations in Excel enabling you to gain valuable insights from your data with ease so let’s move to Advanced analysis and automation so lastly we will explore Advanced analysis and automation so Excel capabilities extend beyond basic functions and chat jpt can help you leverage its full potential and for this also we’ll write a prompt to the chat GPD that can you assist me in building a forecasting model to predict future sales based on historical data present in the spreadsheet so let’s see what Chad j respond to us so let’s ask him can you assist me in building a forecasting model to predict future sales based on historical data so let’s see what CH J respond to us so you could see that building a forecasting model to BU future sales based on historical data involves several steps so for this you have to prep prep the data select a model train the model evaluate the model and make predictions so let’s start by preparing the data we’ll aggregate sales by order date to create a Time series of total daily sales then we’ll choose a model based on data’s characteristics so we will grant the permission to chat gyy to proceed with the steps so we started analyzing the data so you could see that that the data has been prepared into a Time series of total daily sales and before choosing a forecasting model it’s helpful to visualize the sales time series to understand its characteristics such as Trend and seasonality so Jad jyy has looed a daily sales data and then he has given the visual Trends and the possible seasonal patterns and the next steps would include testing for stationarity model selection and parameter touring model training and then forecasting so you could follow all these steps with the help of chat GPT and could create a forecasting model if you want we can move to the next step so you could like provide a prom to the chat chipy that move to or proceed with Next Step so step by step JJ will provide you a forecasting model or it will help you create a forecasting model as you can see it is started analyzing it so the augmented D Fuller that is ADF test gives up a P value of approximately 0.33 since this P value is less than the common significance level of 0.05 we can reject the null hypothesis that the time Series has a unit root meaning the time series is stationary so you could see that is asking us shall we proceed with the Sima model parameter selection yes here is the model parameter selection so you could select any based on the trends and the bar graphs ploted by the chat gbd so you could see guys that the chat G has analyzed the spreadsheet and provided the response that the grid search for the optimal Sera parameters based on the lower information Criterion suggest the best fit model uses the parameter 011 and for ARA and 01112 that is for the seasonal component so you could see that it has provided the model and the parameters now he asking us shall we move on to forecasting future sales with this model yes proceed with that so you could see that with chj assistance you can build complex forecasting models in Excel allowing you to make informed decisions and plan for the future effectively so chat jpt is a valuable tool for Excel users and data analysis offering assistance in data cleaning analysis visualization and Automation and by integrating Chad GT in your Excel workflows you can enhance your productivity and uncover new insights from your data so you can try J gbt for Excel data analytics today and take your Excel skills to the new level and you would see that uh this an error in the analyzing and he has encountered an error while trying to plot the forecast with its confidence intervals so this type of error can occur due to data type issues particularly when handling data indices in forecast and despite the cup the forecasting process using the ca model was completed and we were able to generate forecast for the next 12 months unfortunately due to the plotting issue he couldn’t visually display it so we’ll ask him can you try once again so you could see that this time J gbt has provided us the forecast for the next next 12 months that is the forecast have been successfully ploted along with the historical daily data so here we have our forecast model and with the chat G’s assistance we can move with other analysis also today we are going to talk about Anova in Microsoft Excel and I will walk you through a stepbystep guide on how to perform this in Excel but before we dive into that let’s first take a quick look at what Anova really means Anova which is analysis of variance in Excel is a statistical method to compare the means of three or more groups to see if they are significantly different from each other it helps determine if variations in data due to real differences or random chance it’s useful because it helps in finding differences as it shows if groups have different results it saves time as it tests all the groups at once it clears insights and helps make decisions based on our data for example if teacher wants to compare the effectiveness of the three different teaching methods on students Math course they can divide the class into three groups of 14 students each using Anova the teacher can analyze the math scores to determine if one teaching method is truly better or if the differences in scores are due to chance now that we have a basic understanding of what Anova actually is let’s get into Excel and see how you can easily calculate this using the build tools I have a sample of data in my Excel to perform the Anova over there is an amazing feature in Excel that many people are not even aware about is a data analysis tool pack if you have not come across this word the data analysis tool pack in Excel is an addin that provides the Advanced Data analysis tools for statistical and Engineering calculations it simplifies the complex data analysis task by offering the buil-in features to perform various statistical test and data processing without needing to write formulas manually so we need to ensure first if we have activated the tool pack correctly so for that we need to go to the file and then options and then we have to move to addin and here in the manage you have to choose for the Excel addin and click on go after that you need to click on this analysis tool pack and click on okay now when you see the topmost bar that is the data if you click on this data you will see a data analysis button over here now we are all set to perform the analysis of variance to perform this I will click on data analysis button and from there I will choose an over single factor and select on okay now here in the input range we need to enter the range of the cells containing your data for that I will click on the supper arrow button and highlight mys that containing the data the next option is grouped by depicts how your data is arranged in your sheet if it is arranged in a column like like I have arranged choose columns and if it is arranged in a row go for the rows now here in the labels in the first row if you have labels like mine like group a group b and Group C do take this button then Alpha is generated and here in the output options this is where you want your result to be return the first option is the output range is where you can specify a particular cell where you want your result to be returned the second option will enter the answer in a new worksheet and the third option will enter the result in a particularly new Excel file I will chose the second option and we’ll name it as Anova now we have got this result within just one click you can definitely get the Anover single Factor results now I will explain each factor what it means count some average variance SS dfms Etc in the next PP so the first table depicts the summary of the data available in the analysis so the first one we have got is count count is a total number of data points in each group here in group a group b and Group C we have 14 data points now moving on we have sum sum is a total value if you add up all the values in each group so the sum of group a is 958 sum of Group B is 1093 and sum of Group C is 1280 now moving on we have average average is the sum divided by the total number of data points in each group so the average of group a is 68.42% this is the average of the square differences from the mean now the variance of group a is 14134 the variance of Group B is 20 9.60 and the variance of Group C is 11 9.49 now the second table here is divided into three groups the between groups within groups and total so the first row depicts the result when the between rows are classed as a source of variation the second row depic when the source of variation is with within the groups and the finally the last row is just the total now we will see what each column represents so the first one we have is SS SS stands for sum of squares between group SS is the total variation explained by the differences between the groups within group SS is actually the total variation within the groups due to the random error or individual variability and the total s’s is the sum of the between group s s and within group SS so the between group SS over here is 37351 n within group SS is 6115 78 and the total SS is 98597 now moving on we have DF DF stands for degrees of freedom this is calculated slightly differently for between groups DF is the number of groups present in your data minus one I have three groups so 3 minus 1 I have got two to determine this within groups DF is the number of observations minus the number of groups that is 42 minus 3 that is equal to 39 then we have MS Ms stands for mean Square you can think it as the variance for each source of variation to calculate this we can simply divide the sum of squares by the degrees of freedom now moving on we have f f stands for statistic here the F statistic is calculated as Ms between the groups divided by the MS within the groups and it is equal to 11995 4 now moving on we have the F critic F critic stands for f critical value it is calculated easily by Excel or you can perform it manually by looking at the critical value table so the F statistics is then compared with the F critical value if the F statistical value is greater than the F critical value we can conclude that the test is significant now moving on we have the P value the probability of obtaining an F statistic as Extreme as were more than the observed value assuming the null hypothesis is true a small P value that is less than 0.05 suggests that the significant differences between group exist as my P value is more than the alpha level that is the 0.05 we fail to reject the null hypothesis therefore there is no difference between the means of the three groups so now you can see some data on my spreadsheet so we have row ID order ID audit date and ship date ship mode Etc so this means that this particular data set belongs to a a sales data set for a store or something so what I have done is if you look out the videos on our Channel we already have created one of the interact Ive dashboards using the same data set so we try to extract month- on Monon sales region category subcategory sales country-wise sales shipment type sales Etc right so I have uh you know kept the data as it is and all the pivot tables as it is now let’s say we do not know how to create all these pivot tables and pivot charts and slicers Etc we just know that we have the data and you’re new to this and you want to an analyze your data how do you do that chat GPD is here for the rescue so before we get on to chat GPD let’s quickly remove all our reference pivot tables about pivot table so I’m quickly deleting all these they left out with only the data right and again you can do this dashboard using chat gbd 3.5 or in case if you have a premium version which is chat gbd 4 you can detect ly attach your data set onto the chat GPT prompt and chat GPT will do all the analysis you need right but for now let’s experiment using the chat GPT 3.5 version which is basically open source and free to use now let’s quickly hop on to chat GPD so there you go we have chat GPD version 3.5 we are not switching to four so now let’s quickly um EXP explain what we are up to H GPD just keep it simple as if you’re talking to your friend the D set belongs to Superstore sales or is about the data set is about uh Superstore sales report and I need to create a dashboard for mon one month SE so this is the prompt and basically I just explained in a brief way I’m not uh giving a detailed information but hope this works so basically uh what you are supposed to do is uh give uh the requirement one by one but I’m giving it in a single PR so that I’ll get the results in one single Pro let’s move for that so what you’re trying to explain CH GPD is that you have a sales data set and you’re using Excel for for storing it and for manipulating the data and trying to build some Innovative interactive visually appealing charts and you don’t know how to do it so you explain that to your chat jpd and uh what are the uh things you are looking to build what are the charts you’re looking to build so I’m looking to build a month- on-month sales report which can be a quarter on quarter month on month year on Year date wise however the way you like it right so uh make sure that your data is properly formatted first so here let’s check out that your date column is formatted to date data type right sometimes what happens is the representation of date uh changes to user by user let’s say this particular data set or this particular inventory is been allocated to five different people to entry the data right somebody might use this type of you know here you have uh let’s create a new sheet and try to let me elaborate what I’m trying to explain so let’s say U person one person three so let’s say I use this type of uh formatting 15th December 2025 something like that and somebody else uses 15 12 2025 and another one uses 15 December only 25 and also sometimes few people also uh intend or you know may uh follow uh a date pattern which looks something like this December the 31st 2025 right and it’s not wrong so many people will have different uh conventions of writing their dates so you need to make some data cleaning and okay if you are coming back to data cleaning we have listed a video in our descriptions for data cleaning in Excel and uh that would help you in various uh data cleaning operations that are um you know used in day-to-day lives of a data analyst so that should help so basically what you are looking for is so instead of General go back to the data types and here you can check out the data type options and you can either give a short date long date or in the case you want to go for some additional formatting you can also go to more formatting and here you can check out the custom formatting options here you can give a customized option for now I want to go with date and the date I want to go with is this particular one which gives me date month and yeah click on that and you should be good to go right so make sure that all the uh uh dates in your spreadsheet are following one single date pattern so that when it comes to uh month on month quarter on quarter and earby sales that will help you a lot okay it will just give you a filtered option it will give you year on year first then quarter on quarter next and if you expand the option of quarter on quarter you will have month on month and if you expand the month then you will have date wise so you will have a you know better understanding when we perform this operation in a practical Way in real time and then we have the next type of chart where we want to identify country-wise sales then we have regionwise sales so let’s say I have one country here let’s say I want sales from France and uh I want to know what are the uh sales happening in central region of France North region of France so you’ll have region wise sales there and category wise which category is giving you the maximum profits or the maximum sales and inside the category which is that subcategory or which product is that that which is giving you the highest number of sales you can identify that and based on that you can plan a business uh you know you can get up with a business strategy there and shipment modes so which type of shipment mode which should you invest in much so that you can you know offer better services to your customers so everything comes so this is what I want to um you know um extract from my D set so this is my requirement you have explained this to your chat GPD through the prompt and just enter now chat GPT should help you break down the requirements and chat GPT should help you uh how to you know it’ll get tell you how to uh do the process so here it will tell you the line chart so basically it’s a it’s a it’s a rule that month on Monon sales or quote on quote sales basically wherever the time is involved the line chart is the best to represent your numbers so now um it basically tells you how to uh represent your charts in what or how what type of chart to be selected that’s good but now uh can you help me create create the first chart in Excel there you go so it tells you how to create it and sure you have a column for dates and other corresponding values so basically it is telling you to have a properly mentioned or properly defined metadata or column headers then select the data go to the insert tab choose the line chart okay and let’s say uh you wanted to create an interactive dashboard right so to create an interactive dashboard or to basically create a dashboard you need a pivot tables and you don’t know how to create a pivot table you can ask chat GPD to help you create a a pivot table help me create a private table for the first type for all the types okay or for the data set certainly so it’ll mention you the steps to follow to create a PIV table there you go so what’s the first step highlight the range of cells that contain your data set and including headers and go back to Excel and you can just click on any cell and control a all your T set has been selected or these days you can just select one of the cells and go to insert menu and pivot table and now from the table range Excel will automatically select the entire range of the table you’re looking for or you’re working on and in casee if you are having any doubts on that you can just select all the range and go to the same insert option and select the pivot table click on okay in a new worksheet and there you go you have your new uh say U pivot table let’s delete this now remember we wanted to create the month-on-month sale for the first U pivot table and pivot chart so and remember about the uh date formatting so we’ll be looking on that now we will drag odd date onto rows here and you can see here so this is what I was talking about so Excel has automatically segmented your dates we we have about 10,000 rows in this particular data set and it has segregated all the 10,000 rows into four different years 2020 21 22 and 23 and if you expand the selection of year it has uh segregated again into pter one two three and four right and if you expand P one you have month-on-month sales in quarter one month-on-month sales in quarter two and month-on-month sales in quarter three and so on right so this is why I was requesting you to have a proper data formatting for all your dates right which could be really helpful when you want to extract the quarter on quarter sales and date wise sales and year-wise sales right so now we wanted to find out month- on-month sales right or year on-ear sales so where is the sales yeah here we have the sales parameter just drag that onto the values and there you go now format this into the currency type so the currency is dollars American dollars and you can eliminate the leading zeros here if you don’t want them there you go now you have the whole number here you can also do the same for this you can select the entire table go to pivot table okay let’s check the uh options provided by chat GP so basically we have created a pivot table now we want to create a pivot chart how do I create a pivot chart so again it’s making really clear statements and really clear steps so it’s again making sure that you create a pivot table first right so we have a pivot table and uh highlight pivot table data click anywhere on your pivot table to highlight the entire range insert pivot chart with the pivot table selected go to the insert tab on the ribbon then click on pivot chart and uh open the insert chart dialogue box choose the type you want and uh customize the chart select the chart so customizing the chart we’ll look onto it and Fields if you want to keep a few Fields you can and if you want to eliminate a few Fields you can you can also do that refresh data let’s say why refresh data see charb is so brilliant right so let’s say uh I have the sales data until 2023 and now we are in 2024 let’s say we update 2024 onto this particular stret sheet and now we have that particular data in the original spreadsheet but that is missing here in that scenario what you can do is just go to data and here you will find the refresh all option once you click on this you can find the 2024 data updated here for now we have not added yet but in case if it’s added all you can do is just refresh your sheet and everything will change so brilliant suggestion from chat jpt and save and share in case if you want to share so let’s follow the same steps select any cell in the pivot table select all the sheets go to insert Tab and here recommended charts or go to the charts option and select the chart that you are looking for so we are dealing with uh time so I would go with the line chart just click okay but in case if you want to create an interactive dashboard just go to pivot table analyze and here you have the pivot charts and again it’ll give you the same option just select the line chart here and press okay so you have your line chart over here so again if if you are editing this so you can remove the Legends if you want remove um anything else or you can customize that okay there you go and if you want to edit the name of the chart you can also do that so I will do this as M Sales report okay accordingly I’ll change the sheet name there you go right now quickly we will create the other charts as well so the process is same you can hold control and drag this particular sheet and you have the next sheet so the next uh requirement that we had that we requested chat jpd was the country wise sales so everything will remain the same what changes is okay let’s close this and here instead of order data we will have okay let’s remove this from rows and quarters months right country so we will drag country into rows and there you go now we have countries now country wise so you can uh select this cell go to pivot table analyze pivot chart and you can select the bar graph so it automatically gave you the first suggestion is bar graph click on that and you have the bar graph over here and you can also edit the chart you don’t want you don’t want the Legends you can remove them and edit the name of the chart to Country wise sales go and uh quickly drag and drop the sheet again new pivot table and here you wanted categ wise sales or segment wise sales uh close this particular thing be this and instead of country you’re going to add segment to the Rose and remove country and you have segment wise sales select any cell on the pivot table go to pivot table analyze pivot chart and here you can use any of the available um charts here let me use the final chart or histogram bar chart will be better press okay and there you go you have the bar chart same edit your you know customize your uh chart change the name to segment wise sales hold control and drag the sheet now you have subcat wise sales instead of segment you will be selecting the subcategory you can remove segment and you can see the chart also changed here and again you can do the category wise sales so this was segment wise sales you can quickly change the segment to category and you have it we quick quickly go back to yeah so we have done the category Wise subcategory Wise region wise is remaining and shipment modes so quickly drag or drop the region remove subcategory from the list there you have the region wise sales edit the name and the last one which is about the shipment modes remove region and there you go you can create a new dashboard over here and name it as dashboard you can do some customization to your dashboard like selecting these sales giv it a color there you go now you can quickly copy and paste all these uh charts to the dashboard starting from the month-on-month sales copy paste so I’ll quickly do all the uh pas thingss as soon as possible e e there you go you can also manage the size of these charts so that everything fits onto one sheet so for now let’s try to unpen this so that we have a little better visual appearance okay not a problem and now uh we need some filter right so something which can make it a little more interactive that’s when you add slices and let’s say you don’t know how to add slices back to chat gbd and ask chat GPD how do I add slices to my dashboard there you go so is telling you to uh add everything in a step-by-step way and it also is talking about the inter activity right it’s it is asking you to connect all the slices it it’s telling you to report the connections right if you just add the slices based on one single okay basically you need to select one of your charts here and select a pivot table or pivot chart go to the pivot table a chart associate with a slicer and insert and go to the insert Tab and insert a slicer I I’ll tell you that so basically it it’s telling you in a very beautiful way step by-step manner right and even after adding the slicer it it is telling you to connect all the uh basically all the charts or all the um you know chart dashboards and charts you created at this particular tutorial and connect all the charts to that particular slicer using the report connections option and you can get the interactivity right so in in a very beautiful manner it has explained let’s try the way it has explained to us go to the dashboard select any one of it and uh go to the insert option and here you have have the slicer right so now you have all the slices over here so basically we wanted what type of slices so I have created a month-on-month uh chart so I’ll go with audit date slicer which is here then I have created segment wise sales so I want segment slicer country wise slicer category and subcategory wise slicers region wise so region slicer and do I have anything else else let’s scroll down that’s the last one so where is shipment here yeah shift mode country region category subcategory segment order dates and Shi mode so I will have multiple slices in one go right now you have to arrange your slices in such a way that they are interactive and they they look like they are sorted in one place so let me do that quickly we have some space left here in the bottom place of the dashboard you can also keep your slices there so what basically chat GPD meant was if you try to use the slices here it will you know the changes will reflect only in one of the pivot charts not on all the pivot charts so to get that interactivity to get that connectivity between all the charts you need to do a small setting which is called as report connections so select one of the slices over here right click and here you have something called report connections so when you report connections you have all the pivot charts and pivot tables that you just created you can just select all those pivot tables that you created and press on okay so that all these pivot tables are connected to that particular slicer uh you can just expand this and check if all the pivot tables are selected press okay do the same for all the slices I’ll quickly do that e e so there you go we have interconnected all the charts now let’s say I want to find out the sales that happened in the latest year just go to the filters select any of the dates 2023 and you have the data and if you want to select any particular country then you can also do that I want to go with United Kingdom and in United Kingdom we have all the data here let’s say I want to check out only for the phones there you go now you have an interactive dashboard which will tell you the sales that happened on the date of 16th of November 2023 in the region United Kingdom for phones all right and uh segment is home or office ship and mode with standard class and major sales that happen are from nor region and the categor is from tech technology right and that’s how you create an interactive dashboard using chat GPD in Excel so each and every step has been mentioned very clearly by chat GPD and you have all the data in one place so let’s quickly remove all the filters and there you have the entire dashboard now before we get started let’s look at the website through which we want to pull the data now this is the stock analysis website that we are looking at and we want to pull this particular data from this particular data set so here we have some sample data and this is the table that we looking at now if you expand each and everything from this data set we have a few more couple of tables you know go through that but now let’s focus on this particular data right now we need to switch to Microsoft Excel so here you can see you have a lot of options here and if you see into the data here you have an option that says get data and here you can pull data not just from a website you can also pull data from an existing workbook from a database from cloud or any other if it’s data you can just pull it now that’s the best feature of Microsoft Excel now if you just go to the bottom section here which says other sources and here select the from web option and this will open a par quy window for you but before we get started on pulling the data we need to do one more thing which is about copying the URL which you need to paste it somewhere here let’s go back to the website now I can just copy the URL go back to Excel and here you can paste the URL and press on okay now here you go you have established a successful connection with the particular website now here you have a couple of tables in that particular website now just click on those tables and you can see the sample preview of the data you’re looking at table 0o table one table two Etc just look at the preview and select the data table that you are looking forward to have it in your dat set now let’s say I want to go with this particular one you can just click on load or if you have some transformation to be done let’s say you wanted to categorize this based on name like first name last name or if you wanted to make sure to do some calculations or find out the rate of the product or anything as such mathematical calculation or data transformation that you want to do you can go with the option which says data transformation or if you’re feeling happy with the data that’s currently on your data set then you can just click on load now go with load for time being and it’ll run some queries and your data will be loaded and also just in case if you wanted to know if this particular dat set is really dynamic or not let’s say somebody does some an update on the particular website since this is a stock trading website maybe there are updates u based on per day or based on week right and someone adds some new data onto the website and again you cannot carry out the same operation right in such scenarios you just go to this option which says refresh all and you can refresh the data and automatically the new entries will be added to the same table you can also see some of the queries running right now it is quick so you can see that you can see an option that says query is running in the bottom section of the Excel spreadsheet let’s look at it once again while I press the refresh button run in query background that’s how it runs and you have the updated data table on your Excel spreadsheet imagine combining a Simplicity of excel with the power of python sounds exciting right you must be wondering if it requires a lot of coding what if I tell you that with almost negligible coding with python in Excel lets you perform the Advanced Data analysis you don’t need to be a coder or a developer to use it with python and Excel you can analyze large dat dat sets creates stunning visualizations with libraries like mat ploted and even automate task like report Generation all within excel’s familiar interace it’s a game changer for anyone looking to work smarter and faster with data no matter the coding experience in this video I’ll show you some amazing things you can do with python and Excel from creating data frames to filtering data with the simple queries it’s packed with features that make your work easier but before before we dive in let’s take a quick look at what python in Excel is all about python in Excel is a feature in Microsoft Excel that integrates python directly into the Excel environment allowing users to write python code and execute it within Excel shets this integration combines Python’s powerful data analysis visualization and automation capabilities with excel’s familiar interface now that we know what python is in Excel let’s explore how we can use it so I am inside Excel and you must be looking for python over here so basically you can go to this formula section and see insert python option available over here you can get started just by clicking on the icon or you can just click one cell and you can write equal to py and then press tab now the python mode is activated so as we are working with data the first step is to send the data to that python because it needs to see it so that it can actually work with it to do that you need to select the whole data set for that I’ll select this and now you can see over this section that it is written the data range I mean the sales range and headers is equal to true as we have headers over here now press enter now you can see that it didn’t happen anything only it went to the next line for me to run this we need to press control plus enter Control Plus enter and see what we got we got a data frame over here if not aware data frame is a two-dimensional TBL data structure in Python provided by the Panda’s Library it is similar to an Excel spreadsheet or SQL table consisting of the rows and the columns now take a look at beside the formula button here this is actually the python object and even we have a drop down over here so it is written python output and we have a python object that is the present form and the Excel value now I can switch to this Excel value and we get my whole data available over here this is basically dates so we have imported this data set over here now as it is taking too much space I can go back to the python object and make it compact now if you’re interested in knowing what’s inside this data frame you particularly click on this image and there will be a visualization appearing for you the first five sets over here and the last five data sets over here it is basically the beginning and the last part part of the whole data set this data frame is a fundamental part of a specific python Library that’s called the pandas pandas is like your goto toolkit in Python for working with data it makes the handling tables like spreadsheets or databases like this super easy with this data frame structure so whether you’re cleaning the messy data or analyzing Trends or marging data sets pandas has covered everything plus it works great with the libraries like numai and matl so you can explore and visualize data effortlessly so better you can go and research about it I will talk about this later but guys now I’ll tell you what all cool things you can do with this now I’ll go to another cell then switch to python mode py and tab and then take the reference of this previous data frame and I’ll write dot describe open and close parenthesis and then Control Plus enter see I got another data frame back so as I got a new data frame I’ll switch to this Excel mode value and let’s see it’s a date so we have some information so the total count of quantity we have is 20 the mean is 23.7 and the sales mean is also 168 7.6 minimum is this and everything we have got now we’ll move to python object it basically gives a quick Insight of the data set now it must be planning to perform various calculations using this data set so it’s better we give it a name so that you don’t have to reference to the cell again and again for that you need to go and select the first data frame and then write in the formula bar here just type the name given to it I will give it as DF which represents the data frame before everything and then um you can press control enter so I have already defined the name to this so every time I don’t need to take a reference of this data frame now now basically we can now go and write DF do describe control enter now nothing has changed now let’s talk about the shortcut keys like the quotations now suppose if I go to this Excel value and I want my product details over here so for that I would go here DF do product do describe and control enter now you can see that the total product count is 20 the unit products we have is eight and the top selling product is headphones and the frequency of that headphones are five now keep one thing on mind that now my value for this header is just product if I would be having like product details like having spaces and everything I can’t write product over here I need that third bracket on and third bracket off so keep that in mind unless you don’t have a space in your head and name you can type like this you can write DF do product. describe now there are lots of functionalities in P I repeat now there are various functionalities in the Panda’s Library some of them that all are familiar with is a sum and average so let’s suppose I want the total cells over here so I’ll shift to this python mode and I’ll write DF dot sales dot sum and open and close parenthesis then I press control enter for running it and you can see the total sales has come in the similar way you can even get the average you just have to replace this sum method with mean and then Control Plus enter and now you can see the average value of the sales so you must be thinking right now that this can be done with Excel functions as well so why do we need python don’t worry I have got you covered so let’s take this tutorial a little forward let’s think we were a total sales for each date available over here then for this I’ll switch to a cell and I will activate this python mode and then type DF that is this data frame initially we have dot Group by this is a function in Excel then open parenthesis and we’ll write date then close parenthesis dot sales dot sum just like the previous one we made and then control+ enter and I want an Excel view of this whole series so let’s see this may be loading and you can see the date over here and the corresponding sales it works so wonderfully that you can do data filtering and data analysis in just a few seconds so now I want everything on a

    monthly basis so for that I’ll do I’ll get inside this group by and I’ll write PD dot dot grouper first bracket on and will write key equal to date comma frequency equal to M and close parenthesis and control enter so it has basically given C for February for March for April May and in this way you can get the whole monthly wise sales now I want to give this a name as chart type as and I will press enter now I will write chart Dot Plot xal to date comma y = to sales comma kind equal to line and I will close this parenthesis now press control enter and you will get a very little chart over here have to wait for some time and you can see a great thing very interesting one over here now I’ll show something more cooler than this so I’ll select one cell over here now I will select something over here equal to Python and tab and then I’m using pd. melt start the first parenthesis DF that is a First Data frame comma ID verse that is equal to date comma product close the third bracket then comma value verse equal to third bracket open quantity comma sales and control enter and we’ll see something amazing we’ll go to this Excel View and we can see the date product variable and we have actually merged the quantity and the sales and this is basically the date so with this you can do very amazing things in Excel using python so there are much more than this a cell address is a combination of column letter and a row number that identifies a cell on a worksheet basically each cell will have an Associated column number column letter and a row number so you have to keep in mind it will be always be a column letter and a row number each Excel sheet will have suppose for instance if you take row five rows will be in numbers and the columns will be in letters column a column B column C column D and so on and the numbers will be for the rows so for instance if you take this particular uh cell the cell address is D5 row column D and row five so this is the cell address now let’s move on to the next one now you will be asked what is relative cell referencing and absolute cell referencing in Excel so relative cell referencing is usually the relative references change when a formula is copied to another cell depending on the destination’s row and column okay so this is called relative cell referencing absolute cell referencing irrespective of the cell you know you don’t have to depend on the value of the cell destination there is no change when a formula is copied so this we I’ll explain it in the uh next Slide the basic difference is this type of referencing is there by default in Excel relative cell referencing you just insert the formula you don’t need to put a do dollar sign in the formula whereas absolute cell referencing if you don’t want to change in the formula when it’s copied across cell then absolute cell referencing requires you to add a dollar sign before and after the column and row address so C for instance this is relative referencing you know if you see this this particular cell is having a we put the value as A3 into B3 so what it will do is it will multiply A3 and B3 and it will give the product this is relative cell reference whereas absolute cell referencing is again the same thing but you have to put a dollar sign before the column and the row number so here your specific Ying this as A3 into B2 okay A3 into B2 and it will give you this particular answer irrespective of that it will give you it will calculate what is there in this and it will give you the answer okay now we move on to the next point is basic question when you scroll across an Excel sheet sometimes you have a header on the first row and if you scroll down you will not be able to see the first row so there is a process or the there is a trick to freeze the pains in the Excel so you just need to have the rows locked so how do you freeze pain this is the question so what you need to do is you need to select the view tab go to the view Tab and go to freeze pains it’s a basic thing and you will see this if you see here select the third row and click freeze paines so you just select this and click freeze paines okay so you’re going here and you’re going to the third row and you are going to view Tab and you’re selecting free Spain so what happens is first two rows will be locked okay these two rows will be locked similarly for uh if you want to freeze the columns okay okay it’s the same thing I am going to select this particular column and then click on view and then click fre spin so what happens is the first two columns are locked A and B will be locked and if I scroll right you will always see the uh customer name and category column so these two we have locked and the next will be scroll so this is the function of freezing pains okay now coming on to the next one and you will be asked and you have an Excel sheet or a spreadsheet and you want to restrict you want to protect the data for from anyone or from someone cop from copying the cell for your worksheet so how do you do that you need to select that particular data that you want to protect and then click control shift F and then you will see the protection tab okay you just need to go to the protection Tab and select Lo okay and you will be asked to put a password wherein you will be getting the review tab where you you you need to select the sheet put a password and just save it so I will show you across in one of the examples here so this is particular uh data that you have and you need to protect you select it so here as I said I selected it and then I just just press control shift F so here you see the protection tab click the protection Tab and then click select and then uh locked and then click okay and then you need to go to the review Tab and then click on the protect sheet and you having the you need to select this protect worksheet and contents of locked cells so you’re selecting these and you are protecting the content and you just need to uh put some password okay and then click okay okay so this is how you protect the data on a worksheet and you are selecting this area control shift F and just put up a password and select lock sheet so this is how you protect the sheet from anyone to copy it the cont now the next question that you might be ask is what is the difference between a function and a formula in Excel so you will be asked this question and you need to remember this formula is like an equation which is typed in by the user you are typing it suppose A1 + B2 you are typing this formula in one particular cell okay whereas function functions are predefined calculations it is like a formula which is already there in Excel and you can use it so as it says here it consumes more time and it has to be typed in manually you this is as per the need basis and this is particularly it’s very quick and very efficient you have inbuilt formulas in Excel the sum and multiplication product and various other things that you will come across so this is the basic difference between a formula and a function in Excel okay the now the next question that you might be ask is uh or you need to know basically is the order of operations used in Excel while evaluating the formulas so what is the order you need to remember this acronym pad Mass p d m a s or in other simple terms you can use this also please excuse my dear and Sally so these are the acronyms p e d m a s now let’s see what happens first and you uh in math when you do some calculations you have order of operations okay Bard mass is the one of the order of operations now in Excel similarly in Excel when you put up a formula you Excel will calculate it in a particular order and this is called the order of operations and the Order of Operations is first it will calculate what is there in the parenthesis it will perform this function and then it will see if there are any exponents in it and if it is there it will do that calculation the next and after that it will see if there is any multiplication or division so p e d and then it will see there is any addition okay and then the last one will be the substraction so Excel knows this and you have to know this so that when you put up a formula you will know what is going to happen first and you will not mess up with your formula okay this is the order of operations it is one of the important questions in the interview for a beginner Excel level now the next one is uh how will you write the formula for following multiply the value in a cell A1 by 10 and add the result by five and divide it by two it is quite simple okay see I’m just giving it an example in the slide and you might be ask something else so you will be given a task to calculate or put up a formula for this so so when you are asked multiply the value in a Cell of A1 by 10 so this is particularly thing okay A1 is the particular cell that is being given and you’re multiplying it okay you put it in parenthesis and then add the result by five so it will calculate the parenthesis first okay and if it sees if there are any exponents you have to think in that way if there are any exponents in it or not and then first it will see the parenthesis and then exponent there are no exponents here and then it will do the multiplication and then the addition and the division okay so you will get this particular answer and if you put it simply in this way your answer is wrong because it will multiply it into 10 add 5 and divide by 2 so you have to check which one is right and you have to make sure that you’re putting the right parenthesis okay so this is the formula okay the next question is uh there is a difference what is the difference between count counter and count blank these are three particular important things that you need to these are similar but they are quite different when you actually see it okay so first we will see what is count count is basically it counts the number of cells which have numeric values as the name suggest it is count it will count the number of cells which has numeric volumes okay if you see this column and we at A1 we putting it as count equals to A10 to A2 to A10 so you’re taking this range and you’re finding out what are the number of cells which have numeric value if you see this 1 2 3 4 5 so five cells have numeric values it counts the cells which has numeric Val and the next thing is the counter okay what counter does it counts the number of cells which has any form of content it can be anything it can be a number it can be a alphabet it can be anything it will see what are the uh number of cells which has any form of content okay for example this is the same example we here taken here and we have put the function as counter A2 to A10 now see 1 2 3 4 5 6 seven so it sees that there is seven cells have some content in it it is numbers and letters it can be anything but it has some content so count is the fion for this and the next one is Count blank okay as the name suggests it will count the number of blank cells only so again the same examples count blank A2 to A1 okay it will see what are the number of blanks okay if you see this is blank and this is blank so two cells are blank so it will count the number of blank cells only okay now the next one is uh what is the shortcut to add a filter to a table why do we use filter and it is one of the basics uh function in the Excel where you need to do some sorting and it can be done by using a filter what is the shortcut key it’s very simple control shift and L this is the key that is used to put up a filter you just press this control shift L all together and you will find the filter and you can sorted now the next question that you’ll come across is uh how do you create a hyperlink in Excel so it’s quite simple there is a simple shortcut control K you press these two together and you will get the option you will select the it will select the cell where you have to insert the hyperlink hyperlink basically it redirect to any other document it can be a web page it can be a Word document it can be another Excel sheet or so on okay so these are the things okay now another important questions uh that is uh how to merge multiple cells text strings in a Cell so how do we merge the text strings in multiple cells into one cell there is one particular uh simple function that you can use used which is called called as concatenate concatenate function is used to merge text strings present in multiple CS okay and uh it’s a very simple thing you just need to put this equals to concatenate and then if you for example this is the formula concatenate text one text 2 text three or this is concatenate A1 comma B1 comma C1 this is the data in our uh three cells X cell is one in A1 B1 and C1 so now if you want to put them all in one and you can merge it in one cell using the concatenate function and it’s very simple concatenate in parenthesis A1 comma B1 comma C okay and you will get it all merged in one particular cell okay so this is how you use the concatenate function and you can also use the and operator to combine cell values it is the same thing you can use e and operator instead of commas okay and I will show it to you in the example here okay now we have this I’ll just copy this thing to here okay and let’s make it simple and now I want to put this here equals con concatenate and A1 comma B1 comma c i just close it and I put enter so you see this this all three the different text is merged in one particular cell and it will you can perform various merge functions using the concatenate function it’s a very simple thing and it is a frequently asked questions in the interview okay the next question uh it is very very important one again and very basic as well so how can you split a column into two or more columns you have some particular text or a sentence written or data written in your cell or a column and you want to put them across in two or more or many number of columns so how do you perform this function it’s very simple again you need to select that particular cell so for instance we have it here is India is a democratic country India space is space a space Democratic space country so the it’s a different text here and you want to split this particular sentence into different columns okay you need to select that cell and then go to the data Tab and then choose text to columns there will be an option where in you need to select text to columns okay and you will see this screen okay and you need to select the choose the delimeter here and what we are specifying here is first it is delimer and then we are selecting a delimer as space because we see that that is space here which is very common and we want to split it so you will see a preview here which shows what your output will be like India in one column is a democratic country so the dener is space here and and you need to select this and then click next okay so you will see the options select the columns data format and choose the destination now you see the data Pro and you need to select the destination where you want to put this output as and you select that and then click finish and this one what it does it it it splits the text into multiple columns okay so you see this you have selected this as the output and it starts here India is a democratic country so uh you will come across some data which is you know huge and there are spaces or OBS or anything that is uh differentiated it with and you need to put them in different rows or different columns so this is the function or that you need to use to split the text into different or multiple columns okay okay the next uh question can be very simple one or most widely used in fact and what is the use of vooka and how do we use it so what is this particular thing in Excel vup function is used for looking up a piece of information in a table and extracting some corresponding data or information so you have some set of values in one particular column and you want to retrieve or extract the data corresponding to that particular column or a cell in the whole sheet okay so the syntax is something like this it is you need to go to one particular uh cell where you want the output as and you type the function we look up and you are selecting the value where is the cell exactly where you want the output to be and you’re selecting the table here and the you’re specifying the column index and range lookup so value this indicates the value that you have to look for in the First Column of a table okay and then the table table basically this refers to the table from which you have to retrieve a value we will show this in the example in the next slide and the column index column index it provides the column in the table from which we are to retrieve a value okay you will specify by the column index which one you are looking for the next thing is range lookup which is optional true with uh if you put true it is approximate match which is default and false if it is exact match if you’re looking for the exact match so what is the use of V lookup and how do we do it okay so find the product related to a customer name Richard okay you have this sheet and in this one you want to see what Richard has bought okay what is a product that he has bought and you can do this with the vooka function okay so you just go here and you type in the function V lookup you’re naming it as Richard and first thing is the value it is H4 okay and then the next thing is the table okay table is A2 this this is the table okay and you’re looking for customer name Richard okay and then you’re selecting this the whole column index which is E15 that mean you’re selecting this whole table till 15 okay E15 you’re selecting this and then the next thing is optional which is giving you the uh range lookup okay what it does is it’s optional it is you’re looking for an exact value or an approximate by default it is approximate map so how it performs the function okay so it gives you the answer as scam it goes and look ups for Richard in this particular it will give you the output here and the A2 is the table that it will look for the customer name and it will search for Richard and then it will give you the output as cam now the next thing will be we lookup different from lookup function how different they are we look up lets the user look for a value in the leftmost column of a table so as we’ve seen in the example it uh allows you to look for a data in the leftmost column of the table and Returns the value in a left to right way okay it sorts in left to right way where lookup function lets the user look for data in a row column and Returns the value in the another row or column it will give you the lookup funion in the next row column okay not very easy to use as compared to the lookup function okay as we see uh it’s a bit complicated when you compare it to the lookup function it’s quite simple and it is used to replace the V lookup function okay so this is the different lookup and V lookup function now uh Excel is basically used for reporting mainly used for reporting to management you have to extract different kinds of reports and what are the report formats that are available in Excel this is one of the uh important questions again uh for a beginner level and there are three formats basically compact form outline form table form okay these are the three formats that are available report formats that are available in Excel okay the next question is about a function which is how does IF function work in Excel okay the F function in Excel it actually performs a logical test and you will give a condition it performs the test and returns a value if the test evaluates to true and another value which you again will specify if the test result is false okay for instance we have here a simple example which is shown in the slide and what we doing here is a written record is valid if age is greater than 20 we have age here specified and salary is greater than 40,000 else return the record is invalid so we are giving two conditions and it actually checks and evaluates whether these conditions are met and it will give you an output whether record is valid or invalid based on the two conditions that are specified okay and F22 XIs which is this this uh column we are specifying it should be greater than 20 and G2 to G6 which is the salary column and we are specifying the condition to be greater than 4,000 So based on this two things it will check and perform if these conditions are uh met then it will give the desired result okay so this is the function that is used in Excel IF function okay the next thing is sum IF function okay again the sum function adds the cell value specified by a given condition or a criteria so you are giving a condition and you are actually doing another function which is performing the addition okay it will add the cell values based on a condition again it is shown here the total sum of salary where salary is greater than 75,000 again for example the salary column is G2 to G6 and we are doing a sum IF function what the result will be it will perform the addition of salaries which are greater than 75,000 so it will see which values are greater than 75,000 and then then the the output will be it will be adding these salaries which are greater than 75,000 okay now coming to the next uh slide we will discuss a another important function count F and we’ll find the data I result based on the data that we have and the count F function so we have a slide uh data here and using the co data find the number of days in which the number of deaths in Italy has been greater than 200 okay it’s not uh shown here for some reason but I will show you in the exal sheet okay so this is the data that we have and we will try to find the function so we’ll use the count function and count function is again simple count if you are specifying the column that is the uh source that you’re looking for that is G2 the column G and it is two it starts from two row two G2 till g357 which is the end of the column and what you’re looking for is Italy you’re specifying it in codes Italy and then the next column that it will look for is the E column the number of cases okay here we have the number of cases e22 again we are specifying the range and what we’re looking for is number of cases which are more than 20 so we’ll perform this function and it will show you the output okay I’ll show you to you in the Excel so here is the sheet and I will uh try to get the answer here and the so we performing count F okay count F and you’re giving the brackets parenthesis G2 which is the countries column okay countries column here who has specified the whole range 3 5 7 it is Bally uh a long long list of uh data and you’re specifying the r and what you’re looking for is particularly uh any country you can pick up like uh Algeria okay Algeria and you specify it in codes okay comma then what is the next Target that you looking for is the column e where is it is mentioning the number of cases okay so you specify the column range for the number of cases which is again e and then uh the 3 5 Tri 7 I mean it’s a basically long list of data that you’re looking for and then you are what is the condition that you want to know and you have to specify it at C and what you’re looking for is number of cases which are more than and you hit enter okay so it give you the output as 143 so there are 143 days in which the number of cases were more than 20 so so let’s uh refine our search further let’s see 30 and it gives 140 so another example that you can take is let’s pick Afghanistan Afghanistan and so there are 11 19 days that uh you will see the number of cases were more than 30 and this can be done with the count F function so so the next question is uh about the pivot table so what is a pivot table it’s basically is a summary of table of the data set I mean the data that is there that can be you know multiple number of rows and columns which has different uh data that has to be uh that is there in the reports and you need to analyze those Trends and you need to create a report and basically present it to the management or in a presentation so pivot table is a summary of table you know whatever data is there you can put it in a summarized format they are useful if you have long rows or column that hold values you need to track okay so you have long list of data and you need to summarize it and put it in a presentation or a you need to track the values so it’s very simple you have to select the data and then go to the insert Tab and select fav table okay and then it will show up like this and you need to uh select the table which is a table or range and then you need to specify where you want to put up this private table okay and it’s quite simple and I can show it to you in one of the examples okay so what you need to do is you need to select this and once you generate or as we have seen in the previous slide you need to select this and you need to uh select the output where you want to put the PIV table and then once you do that you will see something like this and you need to select these particular fields and put it in value so it will for example you need to select the death number of death you can just simply click this select this drag and drop it in the values okay so these are the different fields that can be put up in the table okay we can show it to you in the example here so this is our data and we need to show it in a p table okay select this and then go to the insert Tab and click private table okay select that and it will show you the uh table or the range and now where you want to put it okay I will put it in the existing worksheet okay and I’ll just select which color I’ll put it here for example and it will show you this and click okay so this is the table and now you need to select the fields that you want to show in the table for example we need to see the continents the number of cases as per the continents okay I selected and put it in the row and we see the immediately we see it in a table where you see the number names of the contents and now I want to see the number of cases and I simply select it and put it across in the value so this is quite simple and very informative very short and crisp form of presenting the data in a pivate table okay so this is how it is being used and it is being presented you just need to go to the insert tab fa table and it will generate the table and then you can select the fields all right so the next question here is uh how do we create a dropdown list in Excel this is particularly useful when you need to sort uh particular data with the drop- down list where number of uh different variables are there and you need to select one particular uh field and sort the data so you can do this by creating a drop-down list using the data validation option present in the data tab it’s quite simple you select the data and you go to the data validation tab in the data and click select data validation so once you select that it will ask you to specify a range so you need to select what data you need to sort and put a drop down so you need to select this and click data validation and you select the list option which shows you that there is a different option okay I’ll show it across in the Excel for example what we have here is uh some particular data and we have different variables we need to sort or put a drop- down list so just go to data as in as said in the slides and go to data validation and click data validation now as you click it it will ask you to data validation settings okay and in the drop down go and select list option so it will ask you to specify the source so when you asked uh to specify the source just click this particular column and click okay so here I’m mentioning the source is column B okay B 3 T B4 Okay click okay and then it will give you the option so here you can uh you can put this in anywhere across the Excel and click the option okay okay so this is the way to drop down using the data validation option so in the next example we will see so how do we apply Advanced filters in Excel we use the advanced filter option present in the data tab so this is the scenario wherein you have multiple set of datas and you need to sort it and it’s it’s actually huge data and you need to sort a particular column or a Fe with this criteria and you can select it and specify the criteria and specify a range what you want to uh select or based on what criteria the range can be specified and you can just get it in a single click so this is the option you have to go to data tab Advanced and you have to apply the advanced filter and then you need to specify the list range that is the basically the source and then the criteria wherein you specify the criteria present here I mean you are specifying in this case we will take an example where we are trying to sort the data of the number of cases based on number of deps and in a particular country and we are specifying the output we the copy to field is the output where you will see the output so I’ll show it to you in the Excel so here we have a spread sheet wherein you will see all the uh data covid data that we have and based on the countries and the continents and the number of deaths number of cases and so on okay and it also has different field like the date uh year and month so so here I am going to sort the data which is particularly for Europe and which has deaths of more than 200 in a day so how do we do it go to data advance and the you need to select copy to another location okay and this is the list range so it’s basically given in this countries okay and the criteria range you have to specify this so I am going to select this as my criteria okay and then I have to put the output in one particular cell so I need to select it again let’s say here okay and then I just click okay see it quite simple instead of running through the whole thing we can just do it with a simple function okay so it shows me the deaths which are more than 200 and for which particular country and for which particular uh continents all right so this is basically made very simple with this option the next question is uh you know you need to highlight some particular cells using using a particular criteria in this case we are trying to analyze or highlight those cells where the total sales is more than $5,000 so what we do is we use conditional formatting to highlight the sales based on the criteria it’s a very useful tool in analyzing data and Visually it helps you know you can have a look at the data in quick flash you will see the data and is quite visible using the conditional formatting so how we do that is I will show it to you now so as said in the slide we have the data of the sales that is provided to us and we need to see who has done more sales which is greater than the $5,000 so just go to home and you need to say go to conditional formatting so for that you need to highlight or select this go home conditional formatting highlight okay and you have different options here you select greater than and you specify the value here my criteria is $5,000 okay we can select which color so let’s go for green okay see this you will immediately see the option you don’t have to dig or look in deep into the data and find out which one is more than 5,000 or any particular value okay so conditional formatting now moving on to the next question the index and match function you must be familiar with the index and match function and why it is used basically index uh function is a very powerful tool where in you know you for example you have a table of planets in our solar system and you want to get the name of the fourth planet Mars so you can do it with a formula using the index one fun and at the same time you can use the match function okay with the match function together you can play wonders with it okay match function is another function that is designed to find the position of an item in a range for example we can use the match function to get the position of a word or uh look or in this case we have a name and we can use the index and match function to find the city how we do that is shown here okay it uh it looks quite complicated here when you see the formula but uh I can show it to you in the Excel and explain it further okay so here I have and I want to know where these people are and where they belong to okay so let’s see how it goes as we have seen it and we use the index function okay index okay now when I have to specify the range okay so the range is A2 it shows it selects the name and I want to select it from A2 to a s okay E7 sorry e call on E7 okay so as you see it highlights the whole table okay now what I’m going to do is I want to match it Okay match it with a condition I am I am specifying a condition a 10 why because I have to specify a name here and it will show me the city where he belongs to okay where it shows is a 2 okay if you see A2 it shows the name through A7 I have to specify the range here okay not just one name we are specifying the whole range as as you see if you put the values it will show you the selection okay a and what it should be it should be the exact match so zero denotes the exact match parenthesis close and I am putting another match condition okay what is this then this will give me the output what output is okay B9 okay this is my city B9 9 is the city where I’m trying to find from okay and what it should look for is a column A A1 through E1 okay it you select these this particular row and what it should be it should be the exact match again okay let’s see how it go it will give you an error because you haven’t put the value here okay so let’s see Andrew okay hit enter and you will see the city where he belongs to okay now let’s see where Anna belongs to enter and show Dallas okay so this is the one other function index and match function being used and it can be asked in frequently in the interview questions okay the next question will be uh how to find duplicate values in a column okay there are two ways that you can actually do this and one is uh basically the very simple one which is the conditional formatting and the other one is we have seen this before count if function okay how we do that you need to Simply select the data and go to conditional formatting and select duplicate values and it’s quite simple and there is another way which give you the color coded conditional formatting okay and there is another way that is the count count if function how we do that is again we you have to put another column and put up a count if criteria cont function wherein you see that you are specifying the range and the the value which is being twice or enter duplicate and you have to put up a condition I’ll show it to you in the X so here we have the data and I need to find the duplicate values here so I just select this and as I said it’s quite simple go to conditional formatting form and then go to conditional formatting and select highlight cell rules and simply go down and select the duplicate values okay it will give you with color you want you can simply select red or anything as you see the moment you select you will see the duplicate values highlighted okay so I just cancel this and there is another way then count a function so I’ll put another column where it says duplicate names and I will put up a cter function here okay so let’s see how a cter function works here to check the or find the duplicate names so as I said I will put up a formula or function here for the count F and we’ll drag it to other CS okay count it and uh uh since it’s an absolute uh reference I have to put up a dollar symbol okay my reference here will be H the column here h 2 okay and again dollar because I’m using it as an absolute reference colum through dollar symbol again nine okay so it drag still at the whole it drags still the all the names okay the H9 all right and then what I’ll do is I’ll put a comma in y I want to do is I have to select h two okay this is my criteria I mean it is repeating twice all right and I have to close the parenthesis and if it is more than one okay so this is my formula I just put enter and it should say true so this is the condition and it is true okay I simply drag it the whole thing and whether events is uh repeated yes it’s true Andrew it’s a duplicate entry false me is a duplicate entry and it says true and as you see me and events are duplicate entries so this is how we use the count function and the conditional formatting to sort duplicate entries in the date now let’s move on to the last question for the beginner level and since we have duplicate entry there is also a problem to remove duplicate entri so how can you remove duplicate values in a range of cells okay to Simply do that you can delete the duplicate values in a column by simply selecting the highlighted cells and press the delete button so you go in each column and you see the duplicate values select it and delete it okay and this is one tedious way of doing it and you can also after deleting the values you can go to the conditional formatting choose clear rules to remove the rules from the sheet so this is one way okay the next thing is again a simple button that is there in the data tab that is the is one of the tools present in the data tab which is simply called as remove duplicates it’s quite simple and it’s there in the data tools let me show you in the Excel so here we are and uh we have this data and we need to check and remove the duplicate so as I said go to the data Tab and simply click on remove duplicates and when you click on it it will ask you you want to continue or expand the selection I just say continue with the current selection and remove duplicates okay select all click okay two duplicate entries that were removed previously we saw that events and uh Anna were there Emy were there so that has been removed okay so that’s it for the beginner level and we’ll go to the intermediate level in the next slides choose from over 300 in demand scales and get access to 1,000 plus hours of video content for free visit skillup by simply learn click on the link in the description to know more so here we are at the intermediate level questions and these are like uh higher level or intermediate level compared to the previous things that we have discussed with the basic level questions uh for the Excel interview so let’s move on to the next thing and so there are there are some scenarios where you enter the date or uh some dates in particular data sheet and you need to find out what is the day of the week okay so you might be asking a question how do you find out the day of a week or a particular date so that is the function we we have a specific function and that is the weekday function to find out the day of the week or a particular date okay so as we know these are the seven days of the week and we just put the function as it says equal equal to week day and we need to enter the column or the area or the cell address here so week day is the function and it will give you the which which day it is of the week okay week as it says one it denotes Sunday okay and three is again Tuesday okay so this is the function that is used to find the day of the week so now we move on to the next one and there is a there might be a question what are the wild cards available in Excel so basically Excel supports three types of wild card which are asteris the star it represents any number of characters it can be used in different formulas and all like multiplication and all so it represents any number of characters okay the next one next Wild Card is the question mark it represents one single character okay you just need to memorize this in the back of your mind that it represents one single character okay whereas asteris is it represents any number of characters and the third one which is rarely used in Excel it is the TAA it is used to identify a wild card character that is Tilla ASX question marks in the text okay and it is very rarely used so you need to memorize these three things that the there are three types of wild cards we can see the functions of these in the later sessions okay the the next one is we have a question here what is the data validation illustrate with an example you might be asked this question and you have to know what is data validation what is what is it used for and you you should know how to set up in example okay data validation is a feature in Excel that is used to control what a user can can enter into a cell okay so you have with the data validation you’re not allowing a data kind of data that should be input in the cell okay for example you have the authority uh by setting up the data validation that you can enter either a number or a text or a just number and so on okay using the data validation and where is it available it is in the data tab okay select data validation option present under data tools okay and you just need to select the particular cells where you want to set up the data validation I’ll show you to you in an example so this is how it looks like you just need to go to data tools and click on data validation and it will show you and you have the options here allow and if you click on the drop- down you will see different options like I will show it to you in the example here okay for this particular thing if I want you know this column should have only names and it should have if you insert a number you will be thrown with an error okay you will get an error message saying that this is not a valid option okay so there are some columns where you have to put only names or you have to put only numers so you can set this up with the data validation technique okay you just allow it or customize it as per your requirement so this is how it looks like when you set up an so here we have set up a data validation saying that it should not allow you to put any text okay so you will get an error when you put a number here the value does not match data validation restrictions defined for this cell okay so here as an example I have some data where I am putting the names here and I’ll put the salary here I want to make sure that you know we cannot enter any numbers here okay it should be only text it should have only name so how do I perform that how do I restrict any user to that they should not be able to enter any number and it should only enter the text okay so there is a function okay you just need to go to data validation click data validation and here in the settings I’ll go to custom there are different options here you you can choose whole numbers decimal list date time so you these are the option and I go to custom and I’m giving a particular formula here okay what is that it should be only text okay so it’s a simple thing it should be text and I need to define the range okay so how I’ll Define the range I just select this okay and I select the range okay so I have selected the range here and if you go to input message and error alert you can customize it and here we are taking the default one and this is the input message if you want to set or customize something okay so let’s see how it works okay I have set this and then I try to put a number here and it throws me an error the value entered is not valid because we have set a restriction a user has restricted values that can be entered into the cell so this is how the data validation work okay the next thing is you will be given a situation wherein you have to uh you will be given data and you have to fun perform a function like in this example we are taking a condition where in you will have data and you have to specify the result based on the data you have to formulate okay so given below is a student table write a function to add pass or fail to the results column based on the following criteria you will be given a criteria so this is the student table you’re giving the student names marks and attendance and then based on that you will have to specify a next uh a a formula wherein you will see the result okay based on the pass uh student data you have to put a pass fail results in the column based on the following criteria what is the criteria and for example here in this example what they’re saying is you have to say this the student is passed if his marks is more than 60 and the attendance is more than 75% okay so how we perform this function as it says you have to use the IF function and check with the and condition to fill the results column okay so let’s see this in the example here here so here is the example and we will perform the IF function and check with the and function so how we do that is here okay so I have to put the IF function here okay and check with the and function okay and I have to specify the conditions I have to specify where it should see the or check the conditions okay so we are checking the conditions here and the marks should be in row I mean the column U and it starts with u5 so let’s see u5 okay it will immediately go to column U and five row five okay and then what is the condition we are satisfying here it should be greater than 60 okay and the other condition is uh the column V that is the attendance column okay and what is the condition we are specifying V 5 should be greater than 75 okay and we just close this conditions with the parenthesis and now we have to specify the result whether it is pass or fail pass comma in codes again F okay so let’s see how this works okay okay see this works okay the marks is 50 and attendance is greater than 80 so he is fail because his marks are 50 his attendance must be more than 75 but yes he is he has not scored marks more than 60 so we drag the same formula to the whole thing and you will see the result okay this is quite simple and you will see this formula and using the if and check and check with and condition okay now moving on to the next uh question you might be asked to you know this is simple function wherein you can you will be asked to calculate your age in years from the current date okay so how we do that there is a particular function and it’s very simple use the ear fraction or dated IF function to return the number of whole days between start date and the end date so you specify a start date which will be uh today’s date and the I mean the birth day and then the end day will be today’s date for example okay and you will get to know the age using the year Frack function and dated IF function okay so here is a small example and this is how we use the ear Frack function okay you specify this uh function and then specify the input dates that is the today’s dates and the date of birth and the uh today’s date okay and this is one of the ways to get it and there is also dated F okay this again it is day to-day function and you specify the input cells okay so let me show you an example for the same so here we are and U we have today’s date that is 4th of May and date of bir for anything it could be so I will specify the ear frag okay equals ear rack okay and then I should as soon as you enter you specify the start dat and the end date okay and I will specify the start date as today’s date it picks the column and then I put the comma and then the end date should be this thing okay and just close it and it will give you the age okay so this is the age for example if you want to find out my age I don’t want to reveal my birthday but it’s okay okay let’s see if it calculates I’m 32 years old okay all right so this is one of the ways and and similarly you can use the dated F function shown in the example all right now moving on to the next question you have nested if statements this is very important and you will definitely be uh coming across this question how are nested if statements used in Excel what are nested if statements the IF function can be nested I mean it can be Loop when we have multiple conditions to meet okay it can be nested the false value is replaced by another if function okay you have specified some condition and you are uh putting up a condition and if that condition is not there or not met then if it will be replaced with another IF function so uh the syntax will be like this below is an example so we actually we use two if statements in conjunction okay so here is an example here we are are specifying a condition where in you know uh the result is excellent if it is more than 80 and if it is less than 160 it is bad or average so this is very simple you want perform one if condition and if that is not met it will go for the next if condition so we are nesting two if conditions we are joining to if statements okay uh I’ll show the same example here in the Excel so here we are we have some data and the students and their marks okay and we are specifying a condition wherein uh if the result or marks are more than 80 the result is excellent and if it is less than 60 it is bad or average okay so how we get this is we use two if conditions if statements so uh as we have seen it if if now the first condition is uh our source or the source is B2 okay so b 2 and what I’m specifying is it should be greater than 80 all right I just close this and the result should be excellent okay excellent if it is more than 80 and if this condition is not met it will go to the next if statement okay we are nesting another if statement what is the condition here for the next statement it will check B2 is less than or equal to 60 you can specify any marks all right so if it is less than 60 it should give the result as bad or average okay specify anything and you okay and just close the double parenthesis so let’s see how it works okay see it works like this now let’s pull up the same formula the for the other results okay see if it is more than 90 it is 90 it is excellent so it is checking the first thing if it is greater than 80 it is excellent okay now in this case for Mark it is bad why because it will first see whether it’s excellent or not if it is not excellent then it will check the other condition if it is less than 60 it will give you the a bad result all right and similarly so on for the other marks all right so this is how we use the nested if statements you are adding two if statements the first condition and the second condition with another if statement okay okay when it comes to excel it is basically involves a lot of data and a lot of data to be analyzed and we have some powerful tools which are used in Excel to analyze the data one such uh important tool is descriptive statistics that is used to analyze the data so we might come across a question and you will be given a data or a table and you need to find the descriptive statistics of the columns using data analysis tool okay so you will be given us maybe for example anything and you will have a lot of data you have to use the descriptive statistics option in Excel for the data analysis okay how you do that is you have to add a pack okay okay you should be knowing that you need to add a pack which is called the analysis tool pack which you go to uh file options and then just click on the addin and select the analysis tool pack click uh Excel addin and just click go then it will add the option and then when you select the data you have to go to the click on the data analysis option which will be which will show up when you select the addin or when you add the addin okay then you need to select the descriptive statistics what we need to do is we need to analyze this data and it will show you the values like mean and the you can also customize you can you see you have this data and for you to check each and every row and find the mean the smallest the maximum number and things like that so this can be easily done using the descriptive statistics for the data analysis uh I’ll show the same thing in the Excel how it can be performed so here we are and I’ll show you first how to add the option to it okay go to add and select analysis tool pack as shown in the slide and you need to click go if you just click okay it will not add so just click go and then you need to select analysis tool pack okay it will give these are the different options uh for now now we are selecting analysis tool p and then click okay as soon as you do that you will see the data analysis this option gets enabled when you add that pack so this is the data that you have you need to find or analyze using the descriptive statistics so as you see it click data analysis and click uh select descriptive statistics okay and it will show up the option like this and you need to select the input range okay okay for some reason it’s already there and if you want me to show I will show it to your across you have to select the whole dat okay this is my input range and you need to select this labels in the first row so that you know it doesn’t take the first row okay I’ll show okay and now I’m putting the output in another worksheet okay you can specify or you can pry a new worksheet and these are the options that you can select you can you are getting a summary statistics which is the default and you can select the confidence level that is the mean value and you can set it up by whatever value percentage you want and you are selecting the largest one and the smallest number so all these data will get analyzed and it will open up in a new sheet and just click okay and you will see this so these are the if you see this this is our data paid organic social and revenue so this is uh okay make it more precise okay so with a single click all our data is analyzed okay this is the mean value and this is the minimum is zero maximum is this is the total and the number of count is thousand so you don’t need to scroll down and check what is the largest amount that is paid so this is the largest amount for the paid okay and so on so for the largest revenue generated all these things are generated with a descriptive analys okay descriptive statistics with a data analysis tool so the next question is about a pivot table and this is more refined or more advanced compared to the last oneth that we discussed in the uh beginner level so here you are asked to create a pivot table to find the total cases in each country belonging to their respective continents you are giving you are being asked to uh put up a table or pivot table with two variables basically each country and their respective continent so again we have the same data and we need to set up a pivot table which shows the continents in the countries with the respective continents the number of cases okay so let’s see how we get this in the example so here we are we have the same data it shows the days months year cases countries and the continents okay continents here and so on okay so I am I’ll go to the insert Tab and select the pavot table okay again the same thing I will be asked to enter a input range okay so I select this and since it’s huge I’ll just put the number to be 7777 okay let’s assume now this is our input data and uh where you want the output to be it’s already selected you can put it anywhere I’m putting it here okay just click okay and it will give you the option to select the fee s now what we want is number of cases okay which will show you the value and the countries and territories their respective continents so okay so this is how it looks like and we have to check now it is asking us to find the countries okay so with their respective continents you see this you see Africa as a continent and you will see okay you will see Africa if you just remove this you will see the number of I mean the cases with the respective continents and if you want to include these countries territories where the respective continents you will see as such if you just expand this you will see the countries that are having the cases in the continent Africa okay so country wise sorted with the continent okay this is another Advanced step to get the data or show the data in the prev table okay another question that you might be ask this is a basic and we have already seen this in the previous example that I have shown how do you provide the dynamic range in data source of a pivot table you have to select a particular data source in your input table of for to show up in the piver table so you have to select the input range but how do you create or how do you create or provide a dynamic range in this data source so just need to create a name table to provide a dynamic range so where is the data of your table so as we have seen in the previous example you will be able to select the input range and you just need to put this value any number so that you know it will pick up the dynamic range for your input source for the tape Okay so Now we move on to the next one and is is about about the pivot table again is it possible to create pivot table using multiple sources of data yes yes you can create a paper table from multiple worksheets sometimes you will have different data in different sheet can you create a pivot table for all the worksheet in a single payot table yes you can but there is a condition there should be a common Row in both the tables so you cannot have different types of data there should be a common row okay so then only it will act as a primary key for the first table and the forign key for the second table okay so this is something that you have to remember and you need to create a relationship between the tables and then build the pivot table so you have to analyze what kind of data it is it should have a common Row in both the tables on both the sheets and then you have to establish a relationship between this table and then build the table okay there is a visit for that we’ll see that later now let’s move on to the next question so the next question is again you know with the pivot table uh wherein you will be asked to further refine or uh you know refine your search and show it in the data so here we have with the covid data example again you will be asked to create a pivot table to find the top three countries from each continent based on the total cases using the co again and we have done this before we have created a prev table for this and we need to further refine it showing the top three countries with the number of cases and I will show that same thing in the Excel and we already have created a table and we’ll refine it further to show the top three countries which are having the highest number of accounts in the respective count so here we are we have the uh table uh already with us and I just just click on the show field list it will show me the countries and territories what we are asked is you know to show the countries with the top three that is number of cases so you just need to go here and you need to select the value filter there is a filter which you can select with the value and now I select top 10 and what I need is top three it’s very simple and you just need to know this thing okay and based on the Su of cases okay this is our field already then just click okay and it will show you the top three countries so it’s quite simple and it is very important to know this uh field and how you play with the value of the filters for each particular uh field in the pivot table so it shows you these things all right and now you can refine it further with the continents and all okay so different continent and you can see this so you go there and there is a value filter you can set up all right you have set up a value filter okay okay now there is another one how do you create a column in a pivot table okay like you will be given a situation where you need to add one more column and show it in the pivot table with a particular calculation or with a particular value and you have you don’t have that actual value in your your data but you have to calculate it and show it across in the table so there is another option which is called pivot table analyze and you need to create a calculated field from pivot table analyze option so when you go to your table and then go to pivot table analyze you will see the option to add a calculated field okay just go there and click select calculated field and you need to Define so in this example we are going to Define another column where we calculate the bonus of the sales and there is a particular formula the formula is you put an if and statement and then you calculate the formula you specify what is the bonus and what is the way that you’re calculating the bonus and it will particularly added another column to your table so let’s see how we we do that in the table actually so here we are we have this table and I have inserted a pivot table and I am putting it in output here okay then you will get the table okay now you need to select the customer name and the unit sold unit price saes okay so we’ll add another column to it how we do that is this go to pivot table tools and then you go to analyze Okay click on field items and set okay you need to select calculated field here select that and we are defining a new field here which is the bonus bonus and as I said we will put up a formula for this and we will Define it how we calculate the bonus okay with an if and statement which we have seen before how we uh put up a and if and statement okay now I’m going to say it as a like this sales my sales should be I’ll specify a condition how they will calculate the bonus if sales is greater than 4,000 and the unit sold okay I am trying to put it as the unit sold it also depends on the unit sold is greater than th000 okay for example let’s say th000 a person who is uh able to do uh sales of th 4,000 and the unit sold should be more than th000 then the sales what is this the bonus will be sales into that is the amount of sales he has done into 5% okay let’s say 5% bonus and sales is if it is 2% okay okay we are adding this field bonus and click add okay okay so we have added this and we have put up a calculation for the formula and you will see this in the so this is simple way of adding the column to your existing data and we are doing the calculations and showing it up okay so this is one way go to pivot table analyze and field items and sets you can select this option and add it okay you can customize this as I said with adding different fields this is pretty handy in analyzing the data which does not exist in the table and you add some value to your favor table okay the next uh question will be about slicer you must be familiar with what a slicer is and a pivot table and how do you use it what are the purpose of using it and how do you put up a slider in the pivot table so how does a slicer work basically slicers are used to further filter data in the pivot table suppose you already have some data and it’s for ease that you can do it by just adding a slicer you can select particular uh data in or a field and you can see the output for that particular field that you have chosen in the slicer okay so it’s very simple go to the insert Tab and select slicer under filters okay and then it will be like this okay and here if for example you see this is our period table we have added two slicers to filter our table that is month and countries and territory so this will be the general pivot table that looks like and we have added another two slies that is month and Country so if you go uh for example if you want to see what is the number of cases and some of the deaths for a particular month and for a particular country you can just do it by a single click and it

    will show that particular data only in the table let’s see and do this in our Excel okay so there is a table uh pivot table already created for this and I want to add a slicer so how do I go ahead and do this is I go to the options and insert slicer okay and the moment you click on it you will see what are the options that you want to add you can select different fields and uh select the slicer for it uh for this example we are choosing month okay this is one of our sliders you can just put it here adjust to the and again this will be our slider okay let’s add another slicer here okay and this will be the countries and territories Okay click on that and you will see two simple badges here that you can choose from so if I want to know in the month of Feb how many number of cases were there for particular let’s see the moment I choose the month it will show that particular number of the corresponding table for the whole and now I want to further refine it to a particular country so for example Australia okay so there were total number of 18 cases and zero deaths at the same time if I go for Algeria it is like this so as you see it goes like this okay so this is one of the very important tool that you can use to show the data in a much simplified Manner and ease okay so the next question is about the pivot table again what is the percentage of contribution of each country and continents to the total cases again the same example for our code the covid one and this time we want to show the output as the percentage of contribution for each country and continents to the total cases so we have the same data and we generate a pivot table and we should show it as the sum of the cases uh to be percentage okay and this is how we do it this is our uh data and I’m going to create a pivot table for this okay so it shows the table and here I have to show what is the fields that we need to select okay for E we just selecting the cases and the contr and territories okay so cases to be the values okay okay so this is how it looks like now when I see let me show it to you see it shows the number of cases now if I just want to refine it further and see it as number of percentage percentage as the number of cases okay so how I do it show field list I go here and I go to Value field settings okay then again number format show the value as okay sorry show the value as you need to select this thing percentage of grand total and then just click okay and you will see this to be the percentage so it shows the data in the form of percentage okay this is very simple and you just need to remember as we have seen before show field and then value field settings okay this is the thing that you need to check okay Grand percentage there are other options also and you can select from whatever you want to do it in this case we are showing it to be the percentage okay okay guys now the next question is about very important aspect of presentation when we come to Pivot charts how do you create a pivot chart in pivot tables this is very basic or very you know important aspect of pivot tables that you can represent a data in a in the form of a chart and there are different types of charts which I’ll show it to you in the uh example okay so we have a data here and we want to show it in right now it is in a table form okay we have put up a pivot table for this and now the same thing can be put up in a fav chart okay so what you need to do is go to the insert Tab and select the pivot chart it depends on different versions of excel that you have different uh forms or different buttons so I will show it across to you and this is how it looks like okay it’s very simple it can be a pie chart it can be a bar chart it can be different forms and we can see that in the example here okay so let’s go to the example and so we have the uh pivot table generated here already what you need to do is you need to select the table table and go to insert and see the options there are there is a column chart there is a line chart there’s Pi Bar area scattered and other parts okay there are different forms okay so right now what we do is we will select the column chart here okay you need to select the pivot table and click on the column chart again there are different forms which can be seen uh in some time now so this is the basic form okay and it will show you the countries I mean the it will show it as a form of a chart okay again you can change the chart type you can change it to Pi okay this personally is very good in presenting or doing presentation and you can select different forms different styles okay will also show you the percentage so this is one form of presenting your table into a chart okay again bar type is also there so this won’t suit here in our example there are different ones I’m just showing it to you as how it looks like so depending on your I mean the aesthetic you can see and select what kind of chart is best suitable okay so this is how you put up a pivot chart in pivot tables okay so this is then last for the intermediate level and what are macros in Excel create a macro to automate a task basically you have some daily task that you perform in Excel okay and you can do this with quite ease using the macros okay it basically is a program that resides within the Excel file and it is used to automate task in Excel some daily task that you run like run when you come across in any data and you have to scrub that data and you do some daily routines and by removing some columns or adding some formulas and you know do doing some adding some colors and changing the fonts or beat anything that you form and you have to do a repetitive task so all that can be done by recording a macro and this is what macro is used for in Excel so how you capture those steps in Excel is through a macro to record a macro you can either go to the developer tab in the Excel and click on record macro or you can use or access it from The View tab Okay so these are the uh ways to access macros okay just go to the developer tab and click record macro where in you know when you click record macro it will record the steps that you performing in each step and it will capture that and you can save that it is like a daily routine that you run and you can do all these multiple steps maybe n number of steps within a single click by running that macro later on okay so you record macro and run that okay you do it through the developer tab go to developer tab and click record macro and it records all the steps that you are performing okay you need to name it and you just need to perform the actions and it will record the macro and you just save it later on whenever you come across the same daily task that you run you can just click it by running that macro okay B so in a single CLI it will perform all the actions that you have done or steps that you have performed on that particular Excel sheet or any other Excel sheet okay this is very handy when you have similar data and you do the similar task each day okay so this ends the intermediate level and in the next we slides we will see the advanced level so now we discuss the advanced level uh interview questions for excel and when it comes to the advanced level you will come across definitely you will be asked about what if analysis how does it work how does it how does what if analysis work in Excel so what if analysis as the name suggest uh you have to have an analysis done okay with the help of different complex mathematical calculations like formulas and using different formulas and calculations you can experiment with the data and you can analyze if this particular input changes what will be the scenario how will your output turn out to be so as you can see uh it is about different variables to see how your changes would affect the outcome of a situation so this is what uh what if analysis is and the same thing can be done with Excel and how it is done is you need to go to the data Tab and click on the what if analysis under forecast okay so there are three options and one is the scenario manager goal seek and the data table so these are the three situation or three tools which are available in the watf analysis in the data tab okay so goal seek is basically for reverse calculations okay and you have some idea you have some set goal in mind and you have certain variables and what you need to do and what you need to achieve to have certain set goals okay you have you will set a value for the goal and with the goal seek option you can get to know what value you have to achieve to come to that level okay so this is what goal seek is it is one of the simplest sense sensitivity analysis tool and uh so for example you know that you know a single outcome you would like to achieve like you have a set goal that you have to achieve this particular grade okay and the goal seek feature in Excel allows you to arrive to that goal by mathematically adjusting a single variable within that equation so you set the goal and you will do the what if analysis goal seek uh option to know what you have to do or what you have to achieve for the set goal the next option is the data table and it is basically uh used for sensitivity analysis okay so in this uh scenario you the data tables allow the adjustment of only one or two variables within a data set but uh each variable can have unlimited number of possible values okay so basically they are used for side by side comparisons and it makes it very easy to read the scenarios okay one but you need to set up the data table correctly okay and what you do is uh you give the input in the table and then you set up the formula and for instance you need to calculate the monthly payments for a loan okay you provide the principal amount like uh whatever amount you’re taking 2 lakhs three lakhs and you set the interest rate in a formula okay and uh you set the term in and then you need to uh provide different uh input options like what loan amount that you’re taking and you set the formula and go to data table and I mean go to what if analysis and select the data table stretches across and then you will see the monthly payment options or the monthly Emi that you can calculate there are different uh things that you can calculate you can calculate the Target or the sales bonus or things like that okay giving different scenarios in the variables okay but you need to set up a data table it might be a little challenging but yes it is a very powerful tool okay so the next one is the scenario manager it is a bit more complicated compared to the other two but then it is uh more advanced than goal seek as it allows you the uh to adjust the multiple variables at the same time okay and uh this is very complicated compared to the other two and it is even I mean it gives more better output when it compares to the other two wtif analysis tools so here when you go to wtif analysis and the scenario manager you have to select the data and provide different scenarios that is different values for each scenario and then it will create a it will analyze calculate and create different scenarios okay so it becomes very easy for you to analyze what if you are changing the value of your interest amount or the uh term of loan or your targets like uh monthly targets or an yearly targets and things like that and you just have to uh see what the scenario will be when you set this particular value for a table or for data whatever targets that you need to set you can use this what if analysis in Excel for the different scenarios and different calculations okay these are very powerful tools and you need to know this now as we move on to the next one you might be asked what is the difference between a function and a sub routine in a VBA okay even though these two are used us quite frequently that is uh pretty much different from each other okay so a function always returns a value of the task it is performing okay so when you are uh writing a module you will have a function when you have a function it will always give you a value okay for example you are performing an addition it using the function it will give you a value for sure whereas sub routine it does not return a value of the task it is performing functions are called by a variable okay you can uh set up a variable and you can call the function okay where sub routines can be recalled from anywhere in the program in multiple types okay in many different ways you can recall a sub routine like we have seen uh we write a sub routine and we put up a button and we can when you click the button it will recall the sub routine functions are called by variable okay you have have to set up a variable and when you enter it it will come up okay functions are used directly in spreadsheet as formulas whereas Subs cannot be used directly in spreadsheets as formula okay this is very important difference and the next thing is the functions can be used to perform repetitive task and return a value whereas users must insert a value in the desired cell before getting the result of the sub so this is particularly limited okay whereas functions can be used to perform different repetitive task and return a value user input is not required whereas if you see the sub routines user must insert a value okay in the desired cell that to in a particular cell which is defined in the sub routine to getting the desired result okay so these are the differences between functions and sub routines now the next one is what is the difference between this workbook and active workbook in VBA so when you work on modules and different Excel spreadsheets and workbooks you will have these different workbooks open and there is a particular difference and you might be asked what is the difference between this workbook and active workbook so as the name suggest this workbook refers to the name of the workbook where the code is actually running from okay you are running a code VBA code and it is the actual workbook where you are running the code from and that is this workbook okay and you have multiple spreadsheets open and you’re working on particular suppose for example you’re working on uh sheet X okay and you active workbook is the workbook that is currently active from the different open workbooks you have different open workbook but you are working on one particular workbook and that is called the active workbook so this workbook is the refers to the name of the workbook where the code is actually written whereas active workbook is the one that is currently open currently active okay so this is the difference between this workbook and the active workbook in VBA as shown in the example this is a simple uh sub that is written a code that can be written to find out which workbook is the uh uh which workbook is the uh this workbook in the VBA and the active workbook as well okay so this is a simple thing that you can run to determine okay it shows the active workbook and the this workbook okay so as we move on to the next one so this is the question how will you pass arguments to a vbf function okay there are two ways basically arguments can be passed to a VBA function as a value or as a reference so this is uh an example that you can see it is by reference the key word for this is by reference okay so when you are passing an argument by reference the keyword should be by RF whereas uh by value your specifying a value and the keyword is by V value okay so these are the two ways that you can pass an argument in VBA so when you pass an argument by reference in a procedure the variable itself okay the variable itself is accessed by a procedure and it is to the location or the address in the memory okay the value of the variable is changed permanently by the procedure in this case all right okay so to pass an argument by reference you should use by ref that is the keyword before the argument okay so in this example it is clearly shown what is the keyword that is being used okay and by reference is the default in the VBA unless you specify something else which in this case is by value okay and as I said the keyword for by value passing the argument by value is by V okay so when you set the keyword by Val the variable uh what happens is the by value argument function or argument uh uh is passed through by value okay the number function is called here which means it assigns a value to the variable X here okay because the variable was passed by value in the function any change to the value okay any change to the value of the variable is only the current function so it ends after it performs the function okay then it revert back to the value when it is declared again where it is set to zero so if the variable has been passed by reference in the function then the variable would have permanently assumed the new assigned value so this is the basic Difference by reference and by value so now we move on to the next one so there might be a question that you will be asked that how do you find the last row or a column in a VBA okay so to find the uh last row sometimes you have long list of uh data and it has like n number of rows and N number of column so instead of scrolling down all together to find the uh last row you can use this uh VBA code you can write this code it’s a very simple one and it will give you the last row you will find the last row with a single click okay let’s see it in the VBA so here I have a sheet which uh you know it has long list of uh rows and instead of scrolling down I can just find out the last row with this VBA code okay I just click on this and as seen in the slideshow I have just return it and you will see the you just run it and you will see the number of it will give you the last row in the sheet okay similarly we can do this for the column as well okay in case if you have long list of or large number of columns instead of scrolling right you can just uh run this VBA and you can see you can know the last number of column okay number of columns so as we’ve seen in the last row now we’ll see the code for the finding the last column it’s the same thing uh instead of row you will see it as column all right and it will when you run it you will give the it will give you the last column okay today I am here to show you how you can start earning extra money with Excel even alongside your 9 to5 job with just an hour a day you can create a steady second income stay tuned as I work you through a stepbystep guide to turning your Excel Sals into a profitable side hustle imagine turning your Excel CS into a steady source of extra income in today’s world where costs are always going up having a second income is not just nice it’s a smart move with just your Excel knowledge you could earn up to 30,000 or more every month helping you manage expenses and chase your dreams you don’t need to be a pro to start beginners can join in too whether you are freelancing analyzing data or creating custom Excel Solutions the possibilities are endless with practice and consistency you can grow your income while building your skills don’t let your Excel knowledge go to West use it to create a flexible financially secure future Excel is not just a tool it’s your key to success and freedom let’s get started and make your skills work for you suppose I have a data set like this and just imagine you have received this from the client’s end so the work will basically be on data analysis and you can have five 00 rows or you can even have 5,000 rows and even have more than that or even less than that so don’t worry it won’t be a manual work for sure with the help of AI that is artificial intelligence we can analyze data in Excel in just a short time now you must be feeling how much they are earning by performing the data analysis in Excel so I have opened here Fiverr if not a is one of the top freelancing platforms that is available and here you will find the various gigs out there when you search for this Excel data analysis now if you look nicely you will see people charging 2,695 and even much more than that and sometimes even less now if I open something like this that is a top rated one you will see an overview see this is the basic there charge for 2,695 and here I will do excel dashboard development and Excel data analysis so if you see minutely you will even have this comments over here they have got from their clients and even they have their overview they what they’re doing uh and what he is building like we build templates off and everything this is a standard one uh he is charging for 8,984 for the um 3-day delivery and n divisions he has done and premium is much more you can even imagine that is 22458 so in a nutshell on an average if you see this whole one you will find that you can earn 1,615 just an hour with your Excel skills now if you open one of the ranking freelancer any of this you will get an overview now I will cross this and I will go back to this data set now we have columns like population on males population on person females literacy rate females and everything and so on so just assume that the client’s requirement was give me the top five City by the average literacy rate females as a pie chart and he will pay you a certain amount for that so the basic part is the client may be un aware of how to do it or maybe having lack of time to do the data analysis in Excel so what you have to do is you have to select this whole data set you have to select this whole data set it will take time as it is so much long and go on the top right side of this Excel and you will find analyze data Under This Home tab so you click on this analyze data and it will take some time for processing after clicking it it will process the data no human interference is required it has automat ically discovered some Data Insights as you can see it has even made this bar graphs and somehow plotted this and even you can get this insights now simply what we need to do is we need to pay attention to what my client requires and simply we need to type that requirement over here and then ask a question about your data and the question to was give me a top five City by average literacy rate females as a pie chart so I will type this question over here so I will type over here the question give me a top 5 City by average literacy rate females as py chart and enter and see I have got one so automatically it will generate a pie chart according to the request M given over here and all we need to do is to click on this insert pivot chart over here so that it will be inserted in a new worksheet sheet two so this is basically nice because you will even getting the five cities that are having the greatest literacy rate of the females and you can earn up to 33,000 with just minimum skill set in Excel with just basic Excel knowledge you can get start a second income stream while your initial earnings might be modest consistent effort and practice will soon help you scale up to 30,000 or more per month as you can experience your skill will command higher pay as a freelancer so I have a pro tip for you start with the competitive rates as a beginner in the freelancing market with time and persistence you will position yourself as a top earning freelancer the journey is worth it your Excel skills can truly transform your financial future so that’s a wrap on a f course guys if you have any doubts or question ask them in the comment section below our team of experts will reply you as soon as possible thank you and keep learning with simply learn staying ahead in your career requires continuous learning and upskilling whether you’re a student aiming to learn today’s top skills or a working professional looking to advance your career we’ve got you covered explore our impressive catalog of certification programs in cuttingedge domains including data science cloud computing cyber security AI machine learning or digital marketing designed in collaboration with leading universities and top corporations and delivered by industry experts choose any of our programs and set yourself on the path to Career Success click the link in the description to know more hi there if you like this video subscribe to the simply learn YouTube channel and click here to watch similar videos to ner up and get certified click here

    By Amjad Izhar
    Contact: amjad.izhar@gmail.com
    https://amjadizhar.blog

  • Algorithmic Foundations of Robotics XI: Collected Papers, Motion Planning, Mapping, Integration

    Algorithmic Foundations of Robotics XI: Collected Papers, Motion Planning, Mapping, Integration

    The provided texts constitute a collection of research papers concerning various facets of algorithmic foundations in robotics. Several papers explore motion planning for single and multiple robots in complex environments, addressing challenges like optimality, collision avoidance, handling dynamic obstacles, and incorporating human guidance. Other works investigate localization and mapping techniques for robot swarms and individual agents under uncertainty, often utilizing probabilistic methods. Furthermore, the collection covers advanced topics such as task and motion planning integration, manipulation in contact, the theoretical underpinnings of robot control, and the application of topological concepts to robotic problems like coverage and knot manipulation. Finally, some papers introduce novel algorithms and provide theoretical analyses of their completeness, optimality, and efficiency in addressing specific robotics challenges.

    Algorithmic Foundations of Robotics XI: Study Guide

    Quiz

    1. What is the primary challenge addressed in “Efficient Multi-robot Motion Planning for Unlabeled Discs in Simple Polygons”? Briefly describe the approach taken to tackle this challenge.
    2. In “Navigation of Distinct Euclidean Particles via Hierarchical Clustering,” what is the significance of hierarchical clustering in the context of multi-agent navigation? Explain the concept of an “admissible cluster.”
    3. According to “Coalition Formation Games for Dynamic Multirobot Tasks,” why are coalition formation games relevant for coordinating multiple robots in dynamic environments? Provide a brief example of a scenario where this approach would be beneficial.
    4. What is the core idea behind “Computing Large Convex Regions of Obstacle-Free Space Through Semidefinite Programming”? How does semidefinite programming help in achieving this?
    5. In “A Region-Based Strategy for Collaborative Roadmap Construction,” how does the approach leverage regions to facilitate the construction of a roadmap for robot motion planning? What are the advantages of this collaborative strategy?
    6. According to “Efficient Sampling-Based Approaches to Optimal Path Planning in Complex Cost Spaces,” what are the key challenges when planning optimal paths in such spaces? Briefly describe a sampling-based technique used to address these challenges.
    7. What is the main focus of “Real-Time Predictive Modeling and Robust Avoidance of Pedestrians with Uncertain, Changing Intentions”? Briefly explain how predictive modeling contributes to robust avoidance.
    8. In “FFRob: An Efficient Heuristic for Task and Motion Planning,” what is the central goal of the proposed heuristic? How does it aim to improve the efficiency of task and motion planning?
    9. According to “Fast Nearest Neighbor Search in SE(3) for Sampling-Based Motion Planning,” why is nearest neighbor search in SE(3) a critical operation in sampling-based motion planning? What makes it challenging, and what is a potential approach to improve its speed?
    10. What is the problem of “Trackability with Imprecise Localization” concerned with? Briefly describe a scenario where a robot might face challenges related to trackability due to imprecise localization.

    Quiz Answer Key

    1. The paper addresses the challenge of efficiently planning collision-free motions for multiple identical (unlabeled) disc-shaped robots within a simple polygon. Their approach involves decomposing the free space and constructing a graph that captures the connectivity of feasible configurations, allowing for efficient path finding.
    2. Hierarchical clustering is used to group the particles and simplify the control strategy by defining collective behaviors based on cluster properties. An “admissible cluster” for a given configuration signifies a cluster where the particles within it exhibit a certain level of consensus, quantified by the non-positive value of $\eta_{i,I,\tau}(x)$.
    3. Coalition formation games are relevant because they provide a framework for robots to autonomously decide which tasks to undertake and with which other robots to collaborate, especially when tasks and robot capabilities change over time. For example, in a search and rescue scenario, robots might form coalitions to cover a larger area or to combine specialized sensing capabilities.
    4. The core idea is to represent the obstacle-free space as a union of large convex regions by formulating constraints on the regions using semidefinite programming (SDP). SDP allows for optimization over convex sets of matrices, enabling the computation of maximal volume ellipsoids or other convex shapes that are guaranteed to be collision-free.
    5. The region-based strategy aims to improve collaborative roadmap construction by having robots independently explore local regions of the environment and then collaboratively merge these local roadmaps based on the connectivity of the regions. This can lead to more efficient exploration and a more robust global roadmap compared to purely centralized or decentralized approaches.
    6. Planning optimal paths in complex cost spaces is challenging due to the high dimensionality of the configuration space, the presence of obstacles or regions with high costs, and the difficulty in efficiently exploring the space to find low-cost paths. Sampling-based techniques like RRT* address this by randomly sampling the configuration space and iteratively connecting these samples to build a graph that converges to an optimal path as the number of samples increases.
    7. The paper focuses on enabling robots to navigate safely in the presence of pedestrians by predicting their future motion and intentions, which are often uncertain and changing. Predictive modeling helps in anticipating potential collisions and allows the robot to plan robust avoidance maneuvers that take into account the uncertainty in pedestrian behavior.
    8. The central goal of FFRob is to provide an efficient heuristic for solving combined task and motion planning problems, which are generally computationally expensive. The heuristic likely aims to decompose the problem or use abstraction to reduce the search space, allowing for faster solutions compared to traditional integrated approaches.
    9. Nearest neighbor search in SE(3) (the space of 3D rigid body poses) is crucial for many sampling-based motion planning algorithms, as it’s used to find the closest existing states to newly sampled states for connection and extension. It’s challenging due to the non-Euclidean nature of SE(3) and the need for metrics that consider both position and orientation. Techniques like specialized data structures (e.g., k-d trees adapted for SE(3)) and efficient distance metrics are used to improve speed.
    10. The problem of “Trackability with Imprecise Localization” deals with ensuring that a robot can reliably track a desired trajectory or goal even when its own localization (knowledge of its pose) is uncertain or noisy. A robot navigating in a GPS-denied environment or relying on noisy sensor data might face challenges in accurately following a planned path or reaching a target location due to imprecise localization.

    Essay Format Questions

    1. Compare and contrast sampling-based and optimization-based approaches to motion planning as represented in the provided excerpts. Discuss the strengths and weaknesses of each approach in the context of different robotic tasks and environments. Refer to specific papers to support your arguments.
    2. Several papers in the collection address multi-robot systems. Analyze the different coordination strategies presented, such as coalition formation, hierarchical clustering, and collaborative roadmap construction. Discuss the conditions under which each strategy is most appropriate and the challenges associated with their implementation.
    3. Uncertainty plays a significant role in many robotic applications. Discuss how different forms of uncertainty (e.g., in sensor measurements, environment models, or agent intentions) are addressed in the featured research. Provide examples from at least three different papers.
    4. The concept of “optimality” appears in several paper titles. Critically evaluate what constitutes an “optimal” solution in the context of robot motion planning and control, considering factors such as path length, time, energy consumption, and robustness. Refer to specific papers that define and pursue optimality in different ways.
    5. Discuss the challenges and advancements in addressing the complexity of robot motion planning in high-dimensional configuration spaces, as evidenced by the variety of topics covered in the excerpts. Consider the role of sampling, abstraction, heuristics, and different representations of the state space in managing this complexity.

    Glossary of Key Terms

    • Configuration Space (C-space): The space that represents all possible poses (position and orientation) of a robot or a system. Each point in C-space corresponds to a unique configuration of the robot.
    • Free Space (Cfree): The subset of the configuration space that corresponds to configurations where the robot is not in collision with any obstacles in the environment.
    • Motion Planning: The problem of finding a valid (collision-free) path for a robot to move from a start configuration to a goal configuration in its environment.
    • Sampling-Based Motion Planning: A class of motion planning algorithms that explore the configuration space by randomly sampling points and connecting them to build a roadmap or a tree, which is then searched for a path. Examples include RRT and PRM.
    • Optimal Path Planning: Motion planning with the objective of finding a path that not only is collision-free but also minimizes a certain cost function, such as path length, travel time, or energy consumption.
    • Multi-robot Motion Planning: The problem of coordinating the motion of multiple robots to achieve individual or collective goals while avoiding collisions among themselves and with the environment.
    • Collision Detection: The process of determining whether a robot in a given configuration intersects with any obstacles or other robots in the environment.
    • Degrees of Freedom (DOF): The number of independent parameters that define the configuration of a robot or a system.
    • Kinematics: The study of motion without regard to the forces causing it. In robotics, it often refers to the relationship between the robot’s joint angles and the position and orientation of its end-effector or other parts.
    • Dynamics: The study of motion in relation to the forces and torques that cause it. In robotics, it involves modeling the robot’s equations of motion, taking into account factors like inertia, friction, and gravity.
    • Heuristic: A problem-solving approach that uses practical methods or shortcuts to produce solutions that may not be optimal but are sufficient for a given set of constraints.
    • Semidefinite Programming (SDP): A type of convex optimization problem involving the optimization of a linear objective function over the intersection of the cone of positive semidefinite matrices with an affine space.
    • Roadmap: A graph representing the connectivity of the free space, where nodes correspond to collision-free configurations and edges represent feasible paths between them.
    • Nearest Neighbor Search: An algorithmic problem of finding the point in a set that is closest (according to some distance metric) to a given query point.
    • SE(3): The special Euclidean group in 3D, representing the space of rigid body motions (translations and rotations) in three-dimensional space.
    • Localization: The problem of determining a robot’s pose (position and orientation) within its environment.
    • Control Policy: A rule or a function that determines the actions (control inputs) a robot should take based on its current state and/or the state of the environment.
    • Stochastic Dynamics: A model of how a system’s state evolves over time that includes random elements or noise.
    • Temporal Logic (LTL): A type of modal logic used to describe and reason about sequences of events in time. It is often used to specify complex mission requirements for robots.
    • Bayesian Approach: A statistical method that uses Bayes’ theorem to update the probability for a hypothesis as more evidence or information becomes available.
    • Gaussian Process (GP): A probabilistic kernel-based model that defines a distribution over functions. It is often used for regression and classification tasks, especially when dealing with uncertainty.
    • Dynamic Programming: An optimization method that breaks down a complex problem into smaller overlapping subproblems, solves each subproblem only once, and stores the solutions to avoid redundant computations.
    • Feedback Control: A control strategy where the control actions are based on the difference between the desired state and the actual state of the system.
    • Lyapunov Function: A scalar function used to analyze the stability of a dynamical system. Its properties (e.g., being positive definite and having a negative semi-definite derivative along the system’s trajectories) can guarantee stability.

    Briefing Document: Algorithmic Foundations of Robotics XI

    This briefing document summarizes the main themes and important ideas presented in the table of contents and selected excerpts from “Algorithmic Foundations of Robotics XI.” The collection of papers covers a wide range of topics within robotics, broadly focusing on motion planning, control, perception, and manipulation in both single and multi-robot systems.

    Main Themes:

    Several overarching themes emerge from the listed papers:

    • Efficient and Optimal Motion Planning: A significant portion of the research focuses on developing algorithms for finding efficient, and ideally optimal, paths and trajectories for robots in complex environments. This includes addressing challenges such as high-dimensional state spaces, kinodynamic constraints, temporal goals, and dynamic obstacles.
    • Multi-Robot Systems: Many papers explore coordination, planning, and control in systems with multiple robots. Topics range from efficient motion planning for unlabeled discs and coalition formation to cooperative roadmap construction and optimal task allocation in delivery systems.
    • Handling Uncertainty and Stochasticity: Several contributions address the inherent uncertainty in robotic systems and environments. This includes predictive modeling of pedestrian intentions, motion planning under uncertainty, active information gathering for localization, and planning in stochastic environments.
    • Advanced Algorithmic Techniques: The papers leverage a diverse set of advanced algorithmic techniques, including sampling-based methods (RRT, PRM), optimization (semidefinite programming, quadratic programming, trajectory optimization), hierarchical clustering, graph search algorithms, and formal methods (LTL, automata).
    • Real-Time and Reactive Planning: Several works emphasize the need for robots to operate in dynamic environments and respond to changes in real-time. This includes real-time motion planning with unpredictable obstacles and robust avoidance strategies.
    • Manipulation and Interaction with Objects: Some papers delve into the complexities of robot manipulation, including orienting parts with shape variation, quasi-static whole-body manipulation, and even knot manipulation.

    Important Ideas and Facts from Excerpts:

    Here are some key ideas and facts highlighted in the provided excerpts, with direct quotes where relevant:

    1. Efficient Multi-robot Motion Planning for Unlabeled Discs:

    • This paper tackles the problem of planning motion for multiple indistinguishable disc-shaped robots in simple polygonal environments.
    • Lemma 9 states: “The combinatorial complexity of D ⋃x∈S∪T D∗(x), is O(m + n).” This suggests an efficient approach to characterizing the free space by considering the union of certain disc-based regions related to start and target configurations.
    • The paper discusses constructing a graph Gi by selecting representative points βi(x) on the boundary of collision discs and connecting start and target positions. This hints at a graph-based approach to solving the multi-robot motion planning problem.

    2. Navigation of Distinct Euclidean Particles via Hierarchical Clustering:

    • This work proposes using hierarchical clustering to navigate a set of distinct particles.
    • It introduces the concept of “hierarchy-invariant vector fields,” defined as: “FHC(τ ) : = { f : Conf ( R d , J ) → ( R d )J ∣∣∣ϕt ( S (τ ) ) ⊂ S̊ (τ ) , t > 0 } , (4)” These vector fields ensure that certain clustered configurations remain within their “stratum” under the induced flow.
    • The paper defines “admissible (valid)” clusters based on the inequality: “ηi,I,τ (x) : = ( xi − mI,τ (x) )TsI,τ (x) ≤ 0 for all i ∈ I . (8)” This condition likely plays a crucial role in the control strategy based on hierarchical structure.
    • The “consensus ball” BQ(x) is introduced as “the largest open ball…centered at c (x|Q) so that for any y ∈ YQ ( x, BQ (x) ) and γ ∈ {σ, τ } every cluster D ∈ {Q,Pr (Q, γ)} \ {P} of γ are partially admissible for y|Q.” This defines a region around a partial configuration where certain admissibility conditions are maintained.
    • “Portal Maps” are defined as a continuous map between different hierarchical structures, aiming to connect different organizational levels of the particle system.

    3. Active Control Strategies for Discovering and Localizing Devices:

    • This paper focuses on actively controlling a robot team to discover and localize devices with uncertain locations.
    • It uses “mutual information” as a metric to quantify the information gained about the device locations through measurements: “MI[x, zτ (cτ )] = D∑d=1 MI [ xd ; zd τ ] = D∑d=1 H [ zd τ ] − H [ zd τ | xd ] (2)” This highlights the information-theoretic approach to active perception.
    • A similar concept of mutual information is applied to discrete grid cells to localize devices within a grid: “MI[g, qτ ] = G∑i=1 MI [ gi , qi τ ] = G∑i=1 H [ qi τ ] − H [ qi τ | gi ] (4)” This demonstrates the adaptability of the mutual information metric to different representations of uncertainty.

    4. Localization without Coordination:

    • This paper presents a distributed algorithm for robot localization that does not require explicit coordination.
    • Algorithm 1 outlines the steps involved, including broadcasting odometry information and then “find θ̂wk |uk , φ̂wk |uk such that (2) holds ∀ j ∈ I“. Equation (2) likely represents a constraint equation based on relative measurements and odometry, allowing each robot to estimate the pose of its neighbors.

    5. Computing Large Convex Regions of Obstacle-Free Space Through Semidefinite Programming:

    • This paper uses semidefinite programming to find large convex regions that are free of obstacles.
    • The method involves finding an ellipsoid E and then iteratively finding a separating hyperplane between the ellipsoid and each obstacle: “We label the point of intersection between Eα∗ and j as x∗. We can then compute a hyperplane, a∗ j x = b, with a j ∈ IRn and b j ∈ IR which is tangent to Eα∗ and which passes through x∗.” This process refines the convex free space representation.

    6. Real-Time Predictive Modeling and Robust Avoidance of Pedestrians with Uncertain, Changing Intentions:

    • This work deals with predicting pedestrian behavior and enabling robots to avoid them robustly, considering the uncertainty in their intentions.
    • The paper uses a probabilistic approach to model motion patterns and assign trajectories to existing or new patterns based on their likelihood: “p(zi = j |t i ,α, θGP x, j , θGP y, j ) ∝ p(t i |b j ) ( n j N − 1 + α ) , (4)” This Bayesian framework allows for adapting to changing pedestrian behavior.

    7. FFRob: An Efficient Heuristic for Task and Motion Planning:

    • This paper introduces an efficient heuristic for integrated task and motion planning.
    • It defines the concept of an “operator” with preconditions, effects (add and delete lists of literals), and a function that maps detailed states: “successor(s, a) ≡ 〈s.L ∪ a.epos \ a.eneg, a. f (s)〉 .” This is a standard representation in planning systems.

    8. Fast Nearest Neighbor Search in SE(3) for Sampling-Based Motion Planning:

    • This paper addresses the challenge of efficient nearest neighbor search in the six-dimensional space SE(3) (rigid body poses), which is crucial for sampling-based motion planning algorithms.
    • It defines a distance metric “DIST Rm P3(q1,q2) = αDISTRm (q1,q2) + DISTP3(q1,q2).” which combines translational and rotational distances with a weighting factor α.
    • The paper introduces a “DynamicKDSearch” algorithm (Algorithm 3) that seems to adaptively refine the search structure based on the query point and the distribution of configurations.
    • The paper discusses splitting criteria for the search structure, including splitting at the midpoint or at a hyperplane intersecting the point being inserted.

    9. Trackability with Imprecise Localization:

    • This paper likely investigates the conditions under which a robot can track a target despite having imprecise localization capabilities.
    • Figure 6 illustrates a “gadget construction” related to intersections and path lengths, suggesting an analysis of how localization uncertainty affects the ability to follow or remain within a certain distance of a trajectory.

    10. Kinodynamic RRTs with Fixed Time Step and Best-Input Extension Are Not Probabilistically Complete:

    • This paper presents a theoretical result showing that a specific variant of the RRT (Rapidly-exploring Random Tree) algorithm, when used with a fixed time step and a best-input extension strategy for systems with kinodynamic constraints, does not guarantee probabilistic completeness (the ability to find a solution path with probability approaching one as the number of samples increases).
    • The problem formulation defines the system dynamics: “ẋ = f (x, u) (1)” and the goal is to find a control trajectory that satisfies these constraints, avoids collisions, and reaches the goal set.

    11. Collision Prediction Among Rigid and Articulated Obstacles with Unknown Motion:

    • This paper addresses the challenging problem of predicting collisions with moving obstacles whose motion is unknown.

    12. Asymptotically Optimal Stochastic Motion Planning with Temporal Goals:

    • This work focuses on motion planning for stochastic systems with goals specified using temporal logic (LTL).
    • It defines the semantics of co-safe LTL formulas over infinite traces: “Let σ = {τi }∞i=0 denote an infinite trace… The notation σ |= φ denotes that the trace σ satisfies co-safe formula φ…” This provides a formal way to specify complex mission requirements that involve sequences of states or events.
    • The problem is framed as finding a policy that satisfies the temporal goal while minimizing risk or cost in a stochastic environment.

    13. Resolution-Exact Algorithms for Link Robots:

    • This paper likely discusses motion planning algorithms for robots composed of links, aiming for solutions that are exact with respect to the discretization resolution.

    14. Optimal Trajectories for Planar Rigid Bodies with Switching Costs:

    • This paper investigates finding optimal trajectories for planar rigid bodies where there are costs associated with switching between different modes of motion or control inputs.

    15. Maximum-Reward Motion in a Stochastic Environment: The Nonequilibrium Statistical Mechanics Perspective:

    • This paper approaches the problem of motion planning for maximum reward in a stochastic environment using concepts from nonequilibrium statistical mechanics.
    • Equation (2) relates the probability of finding a near-optimal path to the expected reward and a concentration term, suggesting a probabilistic analysis of performance.

    16. Optimal Path Planning in Cooperative Heterogeneous Multi-robot Delivery Systems:

    • This paper deals with finding optimal paths for a team of diverse robots (heterogeneous) cooperating to perform delivery tasks.
    • The problem is modeled using a graph with different types of edges representing street and flight movements: “The edge set, E , is a unionof twomutually exclusive subsets, E = Ew∪Ed . The set Ew contains directed street edges… The set Ed contains pairs of bidirectional flight edges…” This graph-based formulation allows for capturing the different capabilities of the robots.
    • The paper mentions a transformation from the Traveling Salesperson Problem (TSP) to their “Heterogeneous Delivery Problem (HDP),” suggesting a connection to classical combinatorial optimization problems.

    17. Composing Dynamical Systems to Realize Dynamic Robotic Dancing:

    • This work explores how to combine different dynamical systems to create complex and coordinated motions, specifically for robotic dancing.
    • Equation (6) defines the desired and actual outputs for the “Single Support” phase of a bipedal robot, relating the robot’s configuration to desired foot placements and joint angles.

    18. The Lion and Man Game on Convex Terrains:

    • This paper likely analyzes a pursuit-evasion game (“Lion and Man”) played on convex terrains, focusing on strategies and conditions for capture.

    19. RRTX: Real-Time Motion Planning/Replanning for Environments with Unpredictable Obstacles:

    • This paper presents RRTX, an extension of the RRT algorithm designed for real-time replanning in environments where obstacles may appear or move unpredictably.
    • Algorithms 2, 3, and 4 describe procedures for adding vertices, culling neighbors, and rewiring the search tree, highlighting the dynamic and reactive nature of the algorithm.
    • Proposition 3 suggests that as the number of nodes increases, the distance between a new node and its parent in the RRTX tree tends to zero.

    20. Orienting Parts with Shape Variation:

    • This paper addresses the problem of manipulating parts with slight variations in their shape to achieve a desired orientation.
    • Definition 2 classifies “p-stable angles” into R-type and L-type based on the behavior of a radius function, which likely characterizes the stability of orientations.
    • Algorithms 1 and 2 outline procedures for constructing critical instances and computing the smallest possible orientation set, suggesting a geometric and analytical approach to solving the part orientation problem.

    21. Smooth and Dynamically Stable Navigation of Multiple Human-Like Robots:

    • This work focuses on enabling multiple humanoid robots to navigate smoothly and maintain dynamic stability.
    • Equation (2) defines the “AVOδ,τ AB” (Avoidance Velocity Obstacle) between two robots, representing the set of relative velocities that would lead to a collision within a time horizon τ, considering acceleration control parameters.

    22. Scaling up Gaussian Belief Space Planning Through Covariance-Free Trajectory Optimization and Automatic Differentiation:

    • This paper tackles the challenge of planning in belief space (the space of probability distributions over the robot’s state) for systems with Gaussian uncertainty.
    • Equations (3a-d) describe the Kalman filter update equations used to propagate the robot’s belief state over time.
    • The paper also presents dynamic models for a two-link manipulator (Eq. 7) and a unicycle robot with sensor noise (Eq. 8), demonstrating the application of their belief space planning approach to different robotic systems.

    23. Planning Curvature and Torsion Constrained Ribbons in 3D with Application to Intracavitary Brachytherapy:

    • This paper focuses on planning paths for flexible instruments, modeled as curvature and torsion constrained ribbons in 3D space, with a specific application in medical brachytherapy.
    • Equation (3) relates the derivatives of the ribbon’s Frenet-Serret frame (tangent, normal, binormal) to its curvature κt, torsion τt, and linear velocity vt.

    24. A Quadratic Programming Approach to Quasi-Static Whole-Body Manipulation:

    • This paper uses quadratic programming to solve problems of quasi-static manipulation involving the whole body of the robot.
    • Equation (1) relates the velocity of the world frame center of mass to the robot’s base velocity and the joint velocities of its manipulators.
    • Equation (4) defines the “base Jacobian,” and Equation (5) relates the center of mass velocity to the joint velocities via the “center of mass Jacobian.”

    25. On-line Coverage of Planar Environments by a Battery Powered Autonomous Mobile Robot:

    • This paper addresses the problem of autonomously covering a planar environment with a mobile robot that has limited battery power.

    26. Finding a Needle in an Exponential Haystack: Discrete RRT for Exploration of Implicit Roadmaps in Multi-robot Motion Planning:

    • This work presents a discrete version of the RRT algorithm for exploring implicit roadmaps in the context of multi-robot motion planning, potentially addressing the combinatorial complexity of such problems.

    27. Stochastic Extended LQR: Optimization-Based Motion Planning Under Uncertainty:

    • This paper introduces a stochastic extension of the Linear-Quadratic Regulator (LQR) framework for optimization-based motion planning under uncertainty.
    • Equations (3) describe the inverse discrete dynamics of the system, and Equation (4) defines the cost function, which includes both path costs and a final cost.
    • The paper outlines an iterative forward and backward value iteration process (lines 6-10 and 13-19 in Algorithm 1) to solve the stochastic optimal control problem.

    28. An Approximation Algorithm for Time Optimal Multi-Robot Routing:

    • This paper develops an approximation algorithm for finding time-optimal routes for multiple robots.

    29. Decidability of Robot Manipulation Planning: Three Disks in the Plane:

    • This paper investigates the theoretical decidability of motion planning for manipulating three disc-shaped robots in a planar environment.
    • The concept of a “stratified configuration space” is introduced, where the space is decomposed into regular submanifolds based on constraints: “Si1i2…im = Φ−1 i1 (0) ∩ Φ−1 i2 (0) ∩ . . . Φ−1 im (0)“.
    • The paper refers to “stratified controllability” as a condition for the system to be able to move in any direction within the configuration space.

    30. A Topological Perspective on Cycling Robots for Full Tree Coverage:

    • This paper takes a topological approach to analyze the problem of using cycling robots to achieve complete coverage of a tree-structured environment.
    • Figure 5 shows simulation results of covering disks, suggesting a focus on the geometric arrangement and movement of the robots for coverage tasks.

    31. Towards Arranging and Tightening Knots and Unknots with Fixtures:

    • This paper explores robotic manipulation strategies for arranging and tightening or untying knots using external fixtures.

    32. Asymptotically Optimal Feedback Planning: FMM Meets Adaptive Mesh Refinement:

    • This work combines the Fast Marching Method (FMM) with adaptive mesh refinement for asymptotically optimal feedback motion planning.
    • Equation (12) represents a discretization of the Hamilton-Jacobi-Bellman (HJB) equation, a fundamental equation in optimal control.
    • Equation (13) and (14) show how the value function at a vertex is computed based on the values at its neighbors in the discretized space.
    • Algorithm 3 outlines a “Characteristic-Driven Edge Selection” process for adaptive mesh refinement based on the value function and its dependencies.

    33. Online Task Planning and Control for Aerial Robots with Fuel Constraints in Winds:

    • This paper focuses on online task planning and control for aerial robots (UAVs) that have fuel limitations and operate in windy environments.
    • The paper mentions a reduction to a Markov Decision Problem (MDP) for planning sequences of discrete states while minimizing fuel consumption and satisfying temporal goals specified by a Büchi automaton.
    • Figure 4 illustrates optimal trajectories for visiting regions while avoiding others, demonstrating the application of their approach to a navigation task with complex temporal requirements.

    Conclusion:

    The collection of papers in “Algorithmic Foundations of Robotics XI” represents a snapshot of the cutting-edge research in the field. The themes of efficiency, optimality, handling uncertainty, and addressing the complexities of multi-robot systems and manipulation are central to many of the contributions. The diverse algorithmic approaches and theoretical analyses presented in these works advance the state of the art in robotic capabilities and provide a foundation for future developments.

    FAQ on Algorithmic Foundations of Robotics XI

    • What are some of the key challenges in multi-robot motion planning addressed in this collection of works? This collection addresses several significant challenges in multi-robot motion planning, including efficiently planning the motion of unlabeled robots (like discs) in complex environments, coordinating dynamic multi-robot tasks through coalition formation, and developing scalable approaches for large teams of robots. It also explores problems related to finding optimal paths for cooperative heterogeneous robots, and handling the complexities of task and motion planning in a unified framework.
    • How are probabilistic methods and sampling-based algorithms being advanced for robot motion planning? The works presented explore various ways to improve probabilistic and sampling-based methods. This includes developing more efficient sampling strategies for optimal path planning in complex cost spaces, addressing the completeness of kinodynamic Rapidly-exploring Random Trees (RRTs), and creating real-time replanning algorithms (like RRTX) that can handle unpredictable obstacles. Furthermore, there is research on asymptotically optimal stochastic motion planning that considers temporal goals and uncertainty in the environment.
    • What role does uncertainty play in the problems studied, and how is it being addressed? Uncertainty is a significant theme, appearing in areas such as robot localization with imprecise sensors, prediction of pedestrian intentions for robust avoidance, and motion planning in stochastic environments. The papers explore methods for trackability with imprecise localization, predictive modeling of uncertain intentions, and stochastic motion planning frameworks that account for state and control-dependent uncertainty, often using Gaussian belief spaces and optimization techniques.
    • How are geometric and topological concepts being utilized in robot motion planning? Geometric reasoning is fundamental, with work on computing large convex regions of obstacle-free space using semidefinite programming and analyzing the complexity of arrangements in multi-robot scenarios. Topological perspectives are also explored, such as in the context of coverage algorithms for tree structures and the decidability of manipulation planning based on the topology of the robot configurations and obstacles.
    • What are some of the novel algorithmic approaches being developed for specific robot types or tasks? The collection features specialized algorithms for various robotic systems and tasks. This includes efficient heuristics for combined task and motion planning, fast nearest neighbor search in the complex configuration space SE(3) relevant for many robots, planning for flexible robots like curvature and torsion constrained ribbons, and approaches for whole-body manipulation using quadratic programming. There’s also work on enabling dynamic robotic dancing through the composition of dynamical systems.
    • How is the problem of multi-robot coordination and task allocation being tackled? Several papers address multi-robot coordination. One approach involves coalition formation games for dynamic tasks. Another focuses on optimal path planning in cooperative heterogeneous multi-robot delivery systems, considering both street and aerial segments. Additionally, there is work on distributed localization algorithms that allow robots to estimate their relative poses without central coordination.
    • What advancements are being made in handling the interaction between robots and dynamic or unpredictable environments, including humans? The research includes strategies for real-time predictive modeling and robust avoidance of pedestrians with uncertain intentions. It also presents RRTX, a real-time motion planning algorithm designed for environments with unpredictable obstacles. These works highlight the importance of adapting plans quickly in response to changes and uncertainties in the environment.
    • How are concepts from feedback control and optimization being integrated into motion planning algorithms? Optimization-based motion planning is a prominent area, with research on asymptotically optimal feedback planning that combines Fast Marching Methods (FMM) with adaptive mesh refinement. There is also work on scaling up Gaussian belief space planning through covariance-free trajectory optimization. Furthermore, the use of control barrier functions and the design of controllers for specific dynamic behaviors like robotic dancing demonstrate the integration of feedback control principles into motion planning.

    Algorithmic Foundations of Robotics XI

    The contents of “Algorithmic Foundations of Robotics XI” represent a cross-section of current research in robotics with a specific focus on algorithms. These algorithms draw inspiration from a variety of classical disciplines, including control theory, computational geometry and topology, geometrical and physical modeling, reasoning under uncertainty, probabilistic algorithms, game theory, and theoretical computer science. A central theme throughout the collection is the validation of algorithms, design concepts, and techniques.

    The field of algorithmic foundations is particularly crucial in the current exciting time for robotics, marked by significant government initiatives and industrial investments. The increasing demand for industrial automation and the development of more capable robotic platforms necessitate the development of sophisticated algorithms. These algorithms are essential for enabling robots and automation systems to operate effectively in complex and unstructured environments. Furthermore, the applications of these algorithms extend beyond physical robotic systems to aid scientific inquiry in disciplines such as biology and neurosciences.

    The research presented in this collection addresses various challenging problems within algorithmic robotics. One such problem is the coordinated motion planning of multiple bodies, specifically fully actuated, first-order point particles that need to avoid self-intersection while reaching a desired, labeled, free configuration. This is tackled using a centralized vector field planner and a hybrid controller, with a focus on computational effectiveness.

    Another key area is hierarchical navigation, which involves planning motion through different levels of abstraction represented by hierarchical clustering. This includes the definition and computation of a “portal map” that serves as a dynamically computed “prepares graph” for sequentially composed particle controllers. The Hierarchical Navigation Control (HNC) Algorithm leverages hierarchy-invariant control policies and discrete transition rules in the space of binary trees to bring almost any initial configuration to a desired goal configuration without collisions.

    The algorithmic foundations also encompass approaches to motion planning under uncertainty. This includes methods that deal with stochastic action uncertainty to achieve high-level tasks specified using temporal logic. Frameworks are being developed to compute optimal control policies that maximize the probability of satisfying such specifications, often by abstracting the continuous stochastic system to a discrete Markov model.

    Different algorithmic paradigms are explored in the sources. Sampling-based algorithms, like Rapidly-exploring Random Trees (RRT) and Probabilistic Roadmaps (PRM), are widely used in motion planning. The collection also delves into subdivision approaches, which offer resolution-exactness by using soft predicates, providing a balance between practical implementation and theoretical guarantees. Exact algorithms represent another approach, though their implementation can be challenging due to numerical errors. Optimization-based planning is also a significant area, particularly for high degree of freedom robots, where trajectories are optimized based on various constraints such as collision avoidance, smoothness, and stability. Additionally, lattice-based motion planning algorithms are utilized for efficient computation of paths in a discretized state space, particularly in scenarios involving maximum reward collection in stochastic environments.

    The algorithmic foundations of robotics also extend to multi-robot systems, addressing problems such as coordinated motion planning and multi-robot task allocation (MRTA). Coalition formation games are used to model the process of finding optimal robot coalitions based on task-related preferences.

    In summary, “Algorithmic Foundations of Robotics XI” highlights the diverse and interdisciplinary nature of algorithms in robotics. It showcases research that aims to develop theoretically sound and practically effective algorithms for a wide range of challenging problems, from coordinated motion and hierarchical planning to handling uncertainty and multi-robot coordination. The emphasis on validation and the exploration of different algorithmic paradigms underscore the ongoing advancements in this critical field.

    Multi-Robot Planning: Concepts and Approaches

    Based on the sources and our previous discussion, multi-robot planning is a fundamental problem in robotics that involves coordinating the motions of multiple robots within a shared workspace to achieve individual or collective goals while avoiding collisions with each other and the environment. This field draws upon various algorithmic foundations, including computational geometry, graph theory, and optimization [Our conversation history].

    Here’s a breakdown of multi-robot planning concepts discussed in the sources:

    • Problem Definition and Complexity:
    • The basic goal is to move each robot from a start to a target position without collisions.
    • This is a natural extension of single-robot motion planning but is significantly more complex due to the increased number of degrees of freedom. Even for simple disc robots, the problem becomes hard when the number of robots is not constant; it has been shown to be PSPACE-hard for rectangular robots and strongly NP-hard for disc robots in a simple polygon.
    • Variants of the Problem:
    • The classical formulation assumes that robots are distinct and each has a specific target position.
    • The unlabeled variant considers all robots to be identical and interchangeable. A generalization of this is the k-color motion-planning problem, with several groups of interchangeable robots.
    • Approaches to Multi-robot Motion Planning:
    • Sampling-based techniques have gained significant attention due to their relative ease of implementation and effectiveness in practice, especially for problems with many degrees of freedom. While single-robot sampling-based methods can be applied to multi-robot systems by treating the group as one composite robot, much work aims to exploit the unique properties of the multi-robot problem.
    • Composite Roadmaps: One approach involves constructing a composite roadmap, which is the Cartesian product of the roadmaps of individual robots. However, the explicit construction can be computationally expensive for many robots. Implicit representations of composite roadmaps are also explored.
    • Discrete RRT (dRRT): This is a pathfinding algorithm for implicitly represented geometrically embedded graphs and can be used for exploration in multi-robot motion planning on composite roadmaps. It reuses computed information to avoid costly operations like collision checking between robots and obstacles by forcing individual robots to move on pre-calculated individual roadmaps.
    • Centralized vs. Decoupled Planning: Centralized planners treat all robots as a single system, while decoupled planners compute trajectories for each robot independently. Sampling-based planners can be used to compare these approaches. Optimal decoupling into sequential plans has also been proposed.
    • Heuristic Methods: Due to the complexity, many heuristic methods have been developed for multi-robot task allocation (MRTA) problems, often viewed as optimal assignment problems considering individual robot constraints.
    • Market-based strategies using auctions are distributed approaches for task allocation, though they can face challenges with remote robots and communication overhead.
    • Coalition Formation Games: This approach models the formation of robot groups (coalitions) to perform dynamic tasks that require diverse resources. A task coordinator is responsible for forming these coalitions based on resource requirements and costs, aiming for stable coalitions where no group has a better alternative.
    • Multi-robot Manipulation: Planning for multi-robot manipulation, especially in cluttered environments, is challenging because the motion of the manipulated object changes the connectivity of the robots’ free space. The Feasible Transition Graph (FTG) is a data structure that encodes object configurations based on robot free space connectivity and transitions between these configurations, providing a framework for complete multi-robot manipulation planning. This approach helps in reasoning about resource allocation, such as the number and placement of robots needed.
    • Multi-robot Routing: The Multi-Robot Routing (MRR) problem focuses on efficiently utilizing a team of robots to visit multiple goal locations without preference for which robot visits which target or the order of visits. While optimal solutions are NP-hard, approximation algorithms with polynomial computational complexity have been developed, along with collision avoidance schemes to ensure robots safely reach their goals.
    • Planning Under Uncertainty in Multi-robot Systems: Stochastic Extended LQR (SELQR) can be extended to plan in the belief space of multiple robots when sensing is imperfect, aiming to minimize the expected value of a cost function while considering motion uncertainty.
    • Graph-based Multi-robot Path Planning: For scenarios where robots move on a graph, such as the pebble motion problem, feasibility tests and planning algorithms have been developed. For example, a group theoretic approach provides a linear-time algorithm for testing feasibility in pebble motion on graphs with rotations (PMR).

    In summary, multi-robot planning is a complex and active area of research with various facets, ranging from fundamental motion planning to sophisticated task allocation and manipulation strategies, often addressing the challenges of computational complexity and uncertainty. The development of efficient and robust algorithms for coordinating multiple robots is crucial for a wide range of applications.

    Fundamentals of Robot Motion Planning

    The motion planning problem is a fundamental challenge in robotics that involves finding a valid (e.g., collision-free) path or trajectory for a robot to move from a start configuration to a goal configuration in its workspace. A configuration describes the robot’s pose (position and orientation) and volume occupied in the workspace. The set of all possible configurations forms the configuration space (C-space). The subset of C-space where the robot is not in collision with obstacles is called the free space (Cfree). The motion planning problem then becomes finding a continuous path within Cfree that connects the initial and goal configurations.

    Here’s a more detailed discussion of the motion planning problem based on the provided sources:

    • Complexity: Motion planning is generally a computationally hard problem. Designing complete planners for high-dimensional systems (more than 5 degrees of freedom) is often intractable. For multiple independent objects, the problem is PSPACE-hard. Even for a single omnidirectional point robot in a 3D environment with polyhedral obstacles, finding an optimal path is PSPACE-hard.
    • Variations and Considerations:
    • Static vs. Dynamic Environments: The basic problem considers static obstacles. However, many real-world scenarios involve moving obstacles, requiring continuous re-evaluation of plans to identify valid trajectories given current and predicted obstacle positions. Planning in unknown environments with obstacles having unpredictable trajectories presents additional challenges, emphasizing the importance of safety and collision avoidance.
    • Kinematics and Dynamics: Motion planning can consider only the geometry (kinematics) or also the motion constraints (kinodynamics) of the robot. Kinodynamic planning seeks trajectories that satisfy both kinematic and dynamic constraints. Some work explores planning with fixed time steps in kinodynamic RRTs, noting that they might not be probabilistically complete.
    • Uncertainty: In many real-world scenarios, there is uncertainty in the robot’s actions and the environment. Motion planning under uncertainty aims to find robust control strategies or policies over the state space, rather than a single trajectory, to maximize the probability of task completion despite this uncertainty. This often involves using Partially Observable Markov Decision Processes (POMDPs) or considering Gaussian belief spaces.
    • Optimality: While finding a feasible path is often the primary goal, optimal motion planning seeks to find a path that minimizes a certain cost function, such as path length, time, or energy. Achieving optimality, especially for systems with dynamics, often requires specialized steering functions.
    • Multi-robot Planning: As discussed in our conversation history, extending motion planning to multiple robots introduces significant complexity due to the increased degrees of freedom and the need to avoid collisions between robots in addition to static obstacles. Different approaches, such as centralized and decoupled planning, composite roadmaps, and graph-based methods, are used to tackle this problem [Our conversation history].
    • Approaches to Motion Planning: The sources highlight several algorithmic approaches to address the motion planning problem:
    • Sampling-based Planners: These methods, including Probabilistic Roadmaps (PRMs) and Rapidly-exploring Random Trees (RRTs), build an approximate representation of the free space by randomly sampling configurations and connecting them to form a graph or tree. While effective in many high-dimensional problems, they can struggle with narrow passages and may not guarantee optimality. Variants like **RRT* ** aim for asymptotic optimality. RRT-connect is an efficient approach for single-query path planning. MRdRRT adapts RRT for multi-robot motion planning on implicitly represented composite roadmaps.
    • Optimization-based Planners: These methods formulate motion planning as an optimization problem, where a trajectory is computed by minimizing a cost function subject to various constraints like collision avoidance and smoothness. Examples include using potential fields, elastic strips/bands, and direct encoding of constraints into optimization costs solved with numerical solvers. Stochastic Extended LQR (SELQR) is used for optimization-based planning under uncertainty. Asymptotically Optimal Feedback Planning combines the Fast Marching Method with adaptive mesh refinement to compute optimal feedback plans.
    • Exact Algorithms: These algorithms aim to find a solution if one exists or report that none exists, often by explicitly constructing the free space or its connectivity. However, they can be computationally very expensive, especially for higher degrees of freedom.
    • Subdivision Approaches: These methods, like the one presented for link robots, use soft predicates and recursive subdivision of the configuration space to achieve resolution-exactness, balancing practicality and theoretical guarantees.
    • Heuristic Methods: Many problems, especially in dynamic or multi-robot settings, rely on heuristic approaches to find solutions efficiently, even if completeness or optimality cannot be guaranteed. FFRob extends the heuristic ideas from symbolic task planning to motion planning.
    • Graph-based Planning: In some cases, the motion planning problem can be abstracted to finding a path on a graph, for example, in the pebble motion problem. Efficient algorithms exist for testing feasibility and finding plans for such problems, sometimes considering rotations of the pebbles.
    • Reactive Planning: These approaches focus on quickly reacting to changes in the environment, often using local planning methods like Artificial Potential Fields (APF).
    • Human-Assisted Planning: Recognizing the strengths of human intuition for high-level scene analysis and the machine’s precision for low-level tasks, collaborative planning strategies like Region Steering allow users to guide sampling-based planners.
    • Integration with Task Planning: For more complex robotic tasks, motion planning is often integrated with high-level task planning. This involves coordinating symbolic reasoning about the sequence of actions with geometric planning of the robot’s movements.

    In conclusion, the motion planning problem is a multifaceted challenge in robotics with significant theoretical and practical implications. The choice of approach depends on the specific requirements of the task, the complexity of the robot and environment, and the need for completeness, optimality, and robustness in the presence of uncertainty and dynamic changes. The research highlighted in the sources continues to advance the algorithmic foundations of motion planning, addressing its various complexities and striving for more efficient, reliable, and adaptable solutions.

    Robot Collision Avoidance Strategies

    Collision avoidance is a fundamental aspect of motion planning, ensuring that a robot can move from a start to a goal configuration without coming into contact with obstacles in the environment or with other robots. The sources provide several insights into different approaches and considerations for collision avoidance in various scenarios.

    Here’s a discussion of collision avoidance drawing from the provided material:

    • Core Requirement: A primary goal of motion planning is to find a path or trajectory that is collision-free. This means that at no point in time along the planned motion should the robot’s physical extent overlap with any part of the obstacle space.
    • Configuration Space (C-space): The concept of C-space is central to collision avoidance. The obstacle space (Cobst) represents all configurations where the robot is in collision, and the goal is to find a path within the free space (Cfree), which is the set of collision-free configurations.
    • Types of Obstacles: Collision avoidance needs to consider different types of obstacles:
    • Static Obstacles: These are fixed in the robot’s workspace. Most traditional motion planning algorithms inherently address avoidance of these by ensuring the planned path stays within Cfree.
    • Dynamic Obstacles: These are obstacles whose position changes over time. Avoiding these requires predicting their future positions and velocities and planning accordingly.
    • Other Robots: In multi-robot systems, robots must avoid collisions not only with the environment but also with each other.
    • Single Robot and Dynamic Obstacles: Several techniques are discussed for avoiding collisions with moving obstacles:
    • Collision Prediction: A novel geometric method is proposed to predict collisions with rigid and articulated obstacles with unknown motion. This approach models obstacles as adversarial agents that will move to minimize the time the robot remains collision-free. The Earliest Collision Time (ECT) is calculated to determine how long the robot can safely follow its current path before a potential collision. This allows for adaptive replanning when a critical collision time is approaching, rather than replanning at fixed intervals. This method can handle arbitrary polygon shapes and articulated objects, overcoming limitations of methods that assume simpler geometries like discs.
    • Stochastic Reachable (SR) Sets: These sets are used to determine collision avoidance probabilities in dynamic environments with uncertain obstacle motion. By formulating a stochastic reachability problem, the probability of avoiding collision can be calculated. Integrating SR sets with Artificial Potential Fields (APF-SR) has shown high success rates in avoiding multiple moving obstacles by using the likelihood of collision to construct repulsion fields.
    • Inevitable Collision States (ICS) and Velocity Obstacles (VO): These are existing concepts where ICS represent states from which collision is unavoidable, and VO are sets of velocities that would lead to collision. These methods often require some information or assumptions about the future motion of the obstacles.
    • Multiple Robot Collision Avoidance: Planning for multiple robots adds significant complexity:
    • Increased Degrees of Freedom: Treating multiple robots as a single system increases the dimensionality of the configuration space.
    • Centralized vs. Decoupled Approaches:Centralized planners consider all robots together, but their complexity grows rapidly with the number of robots.
    • Decoupled planners plan paths for each robot independently and then try to coordinate them to avoid inter-robot collisions.
    • Reciprocal Collision Avoidance (RVO) and Optimal Reciprocal Collision Avoidance (ORCA): These are popular decoupled approaches where each robot computes velocities to avoid collisions, assuming other robots will also react to avoid collisions. ORCA defines permissible velocities as half-planes, leading to smooth and oscillation-free motion. Acceleration-Velocity Obstacles (AVO) extend this by considering acceleration limits.
    • Motion Graphs: For multi-robot planning with unit discs, motion graphs can represent adjacencies between start and target configurations within a connected component of the free space, ensuring collision-free movement between these configurations. The concept of collision discs (D2(x)) defines the area around a robot where another robot cannot be without collision.
    • Composite Roadmaps: For multiple robots, individual Probabilistic Roadmaps (PRMs) can be combined into a composite roadmap (e.g., using a tensor product). This allows for querying collision-free paths for the entire group of robots, and pre-computed individual roadmaps can reduce the need for repeated collision checks with static obstacles.
    • Well-Separated Configurations: Some problem formulations assume that start and target configurations of robots are “well-separated” to simplify initial and final collision avoidance.
    • Human Assistance: In some approaches, humans can aid collision avoidance by providing high-level guidance and identifying regions to avoid, allowing the automated planner to handle the detailed collision checking and pathfinding.
    • Collision Avoidance in Manipulation: When a robot manipulates movable objects, collision avoidance must consider the robot, the object being manipulated, and the environment. This can involve maintaining contact while avoiding collisions.
    • Geometric Representation and Collision Checking: Efficient collision detection algorithms and geometric representations of robots and obstacles (e.g., bounding boxes, collision discs, polygons) are crucial for the practical implementation of collision avoidance strategies.
    • Smoothness and Stability: Collision avoidance is often coupled with the desire for smooth and dynamically stable robot motions, especially for high-DOF robots. Optimization-based methods often incorporate smoothness and stability constraints alongside collision avoidance.

    In summary, collision avoidance is a central challenge in motion planning that requires careful consideration of the environment’s dynamics, the number and complexity of robots, and the desired properties of the resulting motion. Various algorithmic approaches have been developed, each with its strengths and limitations in addressing different collision avoidance scenarios.

    Probabilistic Completeness in Sampling-Based Motion Planning

    Probabilistic completeness is a crucial property for sampling-based motion planning algorithms. It essentially means that if a solution to a motion planning problem exists, the probability that the algorithm finds it approaches one as the running time (or number of samples) tends to infinity. The sources discuss probabilistic completeness in the context of several different motion planning algorithms:

    • Rapidly-Exploring Random Trees (RRTs) and Variants:
    • Standard RRTs are often considered to be probabilistically complete. However, the sources highlight that this depends on the implementation details, particularly how the tree is extended.
    • It has been shown that an RRT using a fixed time step and a randomly selected input (from a finite input set U) is probabilistically complete. However, this variant is often less efficient.
    • The more common variant of kinodynamic RRTs that uses a fixed time step and chooses the best control input to get as close as possible to the sampled state according to a distance metric is not generally probabilistically complete. The provided proof uses a counterexample to demonstrate this. This contradicts the general perception that all RRTs are inherently probabilistically complete.
    • T-RRT and RRT*: Both T-RRT (Transition-based RRT) and RRT* are probabilistically complete.
    • T-RRT*, which integrates the transition test of T-RRT into RRT*, is also probabilistically complete. This is attributed to the probabilistic completeness of RRT*, despite the non-uniform sampling due to the transition test, as the probability of a sample being accepted is never zero.
    • AT-RRT (Anytime T-RRT), an extension of T-RRT, is also probabilistically complete because it behaves like T-RRT before a solution is found.
    • Region Steering: This planning approach is probabilistically complete because it retains the entire workspace as an attract region, assuming that the underlying sampler it uses is also probabilistically complete. If the underlying sampler guarantees asymptotically complete coverage of the space, then Region Steering maintains this property.
    • dRRT (Dynamic RRT for implicit roadmaps): This algorithm is shown to possess a strong property of probabilistically revealing all vertices of the traversed graph (if connected) with high probability, assuming the vertices are in general position. The proof relies on the fact that the random sample needs to fall within the intersection of Voronoi cells to extend the tree, and this intersection has a non-zero measure under the general position assumption.
    • MRdRRT (Multi-robot dRRT): The probabilistic completeness of this multi-robot approach depends on the probabilistic completeness of the underlying single-robot roadmaps and the graph search algorithm (dRRT). While the composite roadmap approach is generally probabilistically complete with a complete graph search, in this case, the graph search (dRRT) is only probabilistically complete, requiring potential refinements to the proof as the Voronoi cell sizes tend to zero. The authors also note that dRRT can be modified to be complete for a finite composite roadmap by systematically exploring unexposed edges.
    • STABLE SPARSE RRT (SST and SST*):
    • SST is proven to be probabilistically δ-robustly complete under the condition that δv + 2δs < δ, where δ relates to the clearance from obstacles, and δs and δv are parameters of the algorithm. This is a weaker form of probabilistic completeness that incorporates a clearance value. The proof involves constructing a sequence of balls covering a δ-robust optimal path and showing that the algorithm has a non-zero probability of making progress along this sequence.
    • SST* is an asymptotically optimal variant of SST that uses a schedule to shrink its parameters over time. It can be proven that SST* is probabilistically complete and asymptotically optimal.
    • Sampling-Based Planners for Temporal Logic: While these methods can quickly find satisfying trajectories for tasks specified in Linear Temporal Logic (LTL), the source notes that they are not correct-by-construction. However, the probabilistic completeness of many sampling-based planners guarantees that if a satisfying trajectory exists, the probability of finding one grows to 1 over time.

    In summary, probabilistic completeness is a desirable property for motion planning algorithms, especially those that rely on sampling. It provides a theoretical guarantee that the algorithm will eventually find a solution if one exists. However, as highlighted by the discussion on kinodynamic RRTs, achieving probabilistic completeness often depends on specific implementation choices and assumptions about the problem and the algorithm’s components. Some algorithms, like SST, offer a δ-robust form of completeness that considers clearance, while others, like SST*, can achieve both probabilistic completeness and asymptotic optimality.

    Partially Admissible Clusters in Hierarchical Particle Systems

    Based on the sources, a partially admissible cluster for a given configuration is defined as follows:

    Definition 3: Let $x \in ( \mathbb{R}^d )^J$, $\tau \in BT_J$ and $K \subseteq J$. Then cluster $I$ of $\tau$ is said to be partially admissible for $x|K$ if $\eta_{i,I,\tau}(x) \leq 0$ for all $i \in I \cap K$.

    To understand this definition, let’s break it down:

    • $x \in ( \mathbb{R}^d )^J$: This represents a configuration of $J$ distinct particles in a $d$-dimensional Euclidean space.
    • $\tau \in BT_J$: This denotes a rooted non-degenerate (binary) tree over the index set $J$, which represents a cluster hierarchy of the particles.
    • $K \subseteq J$: This is a subset of the indices of the particles.
    • $I$ of $\tau$: This refers to a cluster within the hierarchical clustering represented by the tree $\tau$. A cluster is defined as the set of leaves (particles) reachable from a vertex in the tree.
    • $\eta_{i,I,\tau}(x)$: This is a scalar-valued “separation” function that depends on the configuration $x$, the cluster $I$ in the hierarchy $\tau$, and the individual particle $i$. It is defined in Equation (8) of the source as $\eta_{i,I,\tau}(x) : = ( x_i – m_{I,\tau}(x) )^T s_{I,\tau}(x)$, where $m_{I,\tau}(x)$ is the midpoint between the centroids of cluster $I$ and its local complement $I^{-\tau}$, and $s_{I,\tau}(x)$ is the separation vector between these centroids.
    • $x|K$: This likely refers to the partial configuration of $x$ restricted to the particles with indices in the set $K$.

    Therefore, a cluster $I$ of a hierarchy $\tau$ is partially admissible for a configuration $x$ with respect to a subset of particles $K$ if the value of the separation function $\eta_{i,I,\tau}(x)$ is less than or equal to zero for all particles $i$ that are members of both the cluster $I$ and the subset $K$.

    It is also noted that for a partition ${I_\alpha}$ of a cluster $I \in C(\tau)$, the cluster $I$ of $\tau$ is admissible for $x$ if and only if $I$ is partially admissible for all $x|I_\alpha$’s. This highlights that full admissibility can be seen as a collection of partial admissibilities over the entire cluster.

    Configuration Space Strata and Hierarchical Clustering

    Based on the sources, a stratum in configuration space is defined as follows:

    • Given a hierarchical clustering (HC), which is a relation between the configuration space and the space of binary hierarchies ($BT_J$), a stratum is associated with a specific binary hierarchy $\tau \in BT_J$.
    • The stratum $S(\tau)$ is the set of all configurations $x$ in the configuration space ($Conf( \mathbb{R}^d , J )$) that support the same binary hierarchy $\tau$ under the given hierarchical clustering relation HC. In other words, $(x, \tau) \in HC$ for all $x \in S(\tau)$.
    • The collection of all strata for all possible binary hierarchies forms a tree-indexed cover of the configuration space. This means that every valid configuration belongs to at least one stratum.
    • For the specific HC2-means divisive hierarchical clustering method, the source defines open strata ($S^o(\tau)$) and closed strata ($S(\tau)$) based on the intersection of inverse images of a scalar-valued “separation” function $\eta_{i,I,\tau}$. These functions relate to the separation of clusters within the hierarchy for a given configuration.
    • A hierarchy-invariant vector field is defined as one that, when applied to a configuration within a stratum $S(\tau)$, keeps the resulting flow within the open stratum $S^o(\tau)$ for any positive time $t$.

    In essence, a stratum groups together all the robot configurations that are classified as having the same hierarchical clustering structure according to a chosen clustering method.

    Hierarchy-Invariant Vector Fields and Configuration Space Stratification

    Based on the sources, the defining property that characterizes hierarchy-invariant vector fields is their behavior with respect to the strata of a hierarchical clustering. Specifically:

    • A vector field $f$ defined over the configuration space $Conf( \mathbb{R}^d , J )$ belongs to the class of hierarchy-invariant vector fields associated with a binary hierarchy $\tau \in BT_J$, denoted as $FHC(\tau)$, if the flow $\phi^t$ induced by $f$ on the configuration space satisfies the following condition: $\phi^t ( S(\tau) ) \subset \stackrel{\circ}{S}(\tau) , \forall t > 0$.

    This means that if a system’s configuration $x$ starts within a stratum $S(\tau)$ (the set of all configurations supporting the hierarchy $\tau$), then for any positive time $t$, the configuration $\phi^t(x)$ under the flow of the hierarchy-invariant vector field $f$ will remain within the interior of the same stratum, $\stackrel{\circ}{S}(\tau)$. In other words, hierarchy-invariant vector fields keep the system’s configuration strictly within the interior of the stratum corresponding to the initial hierarchy.

    Furthermore, the source notes that any hierarchy-invariant field $f \in FHC(\tau)$ must also leave the collision-free configuration space $Conf( \mathbb{R}^d , J )$ invariant, thus avoiding any self-collisions of the particles along the resulting trajectories.

    Configuration Space Strata in Hierarchical Robot Clustering

    Based on the sources, a stratum in configuration space is a fundamental concept related to hierarchical clustering of robot configurations. Here’s a brief explanation:

    • Given a hierarchical clustering (HC), which defines a relationship between the space of robot configurations ($Conf( \mathbb{R}^d , J )$) and the abstract space of binary hierarchies ($BT_J$), a stratum is associated with a specific binary hierarchy $\tau$ from $BT_J$.
    • The stratum $S(\tau)$ is formally defined as the set of all configurations $x$ within the configuration space that support the same binary hierarchy $\tau$ according to the hierarchical clustering relation HC. Mathematically, $S(\tau) : = { x \in Conf ( \mathbb{R}^d , J ) \mid (x, \tau ) \in HC }$.
    • The collection of all these strata, indexed by the different possible binary hierarchies, forms a tree-indexed cover of the configuration space. This implies that every valid configuration of the robots will belong to at least one such stratum.
    • For the specific HC2-means divisive hierarchical clustering method, strata can be characterized using a separation function $\eta_{i,I,\tau}(x)$. Open strata ($S^o(\tau)$) and closed strata ($S(\tau)$) are defined based on the values of this function for all clusters $I$ in the hierarchy $\tau$. A configuration $x$ belongs to $S(\tau)$ if and only if every cluster $I$ of $\tau$ is admissible for $x$, meaning $\eta_{i,I,\tau}(x) \leq 0$ for all $i \in I$.
    • As discussed in our conversation history, hierarchy-invariant vector fields are characterized by the property that they keep the system’s configuration within the interior of the stratum corresponding to the initial hierarchy.

    In essence, a stratum groups together all the different arrangements of the robots that share the same underlying hierarchical clustering structure as determined by a chosen hierarchical clustering method. These strata provide a way to decompose and analyze the complex configuration space.

    Robot Configuration and Obstacle Space

    Based on the sources, the relationship between obs(x) and the obstacle space O is as follows:

    • The obstacle space O is defined as the region in $R^2$ that is outside of the robot’s workspace $W$. It represents the areas that the robot should avoid.
    • For a given configuration $x \in W$ (where the robot’s center is at point $x$), the obstacle set of $x$, denoted as obs(x), is defined as the set of all points $y$ that satisfy two conditions:
    • $y$ belongs to the obstacle space O: $y \in O$.
    • The Euclidean distance between $x$ and $y$ is less than 1: $|x – y| < 1$. This means that the point $y$ in the obstacle space is within the open unit disc $D_1(x)$ centered at $x$.
    • In other words, obs(x) is the portion of the obstacle space O that overlaps with the open unit disc $D_1(x)$ centered at the robot’s center $x$. Since the robots are considered unit discs, obs(x) contains the points in the obstacle space that the robot would collide with if its center were at $x$.
    • It is also noted that obs(x) is empty ($\emptyset$) if and only if the configuration $x$ is in the free space $F$. The free space $F$ is the set of all configurations $x$ where the open unit disc centered at $x$ does not intersect the obstacle space $O$ ($D_1(x) \cap O = \emptyset$). If $x$ is in $F$, then there are no points in $O$ within a distance of 1 from $x$.

    Therefore, obs(x) is a subset of the obstacle space O that specifically identifies the obstacles in immediate proximity to a robot centered at configuration x (those within a distance of 1). This concept is crucial for defining collisions between the robot and the obstacle space.

    Robot Collision Disc Definition and Significance

    Based on the sources, the collision disc of a configuration is defined as follows:

    For any configuration $x \in R^2$ (representing the location of the center of a robot) and $r \in R^+$, $D_r(x)$ is defined as the open disc of radius $r$ centered at $x$.

    Specifically, the collision disc of the configuration $x$, denoted as $D_2(x)$, is the open disc of radius 2 centered at $x$.

    The significance of the collision disc lies in defining collisions between robots: if a robot is at configuration $x$, then no other robot can be at a configuration $y$ within the interior of $D_2(x)$, denoted as $Int(D_2(x))$, as this would result in a collision. This is because the robots themselves are considered to be open unit discs. Two unit discs collide if and only if the distance between their centers is less than $1 + 1 = 2$. Therefore, $D_2(x)$ represents the region where the center of another robot cannot be located without colliding with a robot centered at $x$.

    Furthermore, the source introduces a related concept, $D^*(x)$, which is the part of the collision disc $D_2(x)$ that is within the same free-space component $F_i$ as $x$. That is, $D^*(x) := D_2(x) \cap F_i$, where $x \in F_i$.

    Well-Separated Robot Configurations: Definition and Implications

    Based on the sources, the condition for well-separated start and target positions is explicitly defined in. Given sets of start configurations $S = {s_1, s_2, …, s_m}$ and target configurations $T = {t_1, t_2, …, t_m}$, where $S, T \subset F$ (the free space), the configurations in $S$ and $T$ are assumed to be well-separated if the following condition holds:

    For any two distinct configurations $x, y \in S \cup T$, we have $|x – y| \geq 4$.

    This condition states that the Euclidean distance between the centers of any two different robots at their start positions, any two different robots at their target positions, or any start and any target position must be greater than or equal to 4.

    Considering that the robots are unit discs (radius 1), this separation distance of 4 ensures that the collision discs of any two distinct start or target configurations do not overlap. Recall from our conversation that the collision disc of a configuration $x$, $D_2(x)$, is an open disc of radius 2 centered at $x$. If $|x – y| \geq 4$, then the distance between the centers of the two collision discs $D_2(x)$ and $D_2(y)$ is at least 4, which is greater than the sum of their radii ($2 + 2 = 4$). Therefore, $D_2(x) \cap D_2(y) = \emptyset$ for any two distinct $x, y \in S \cup T$.

    Furthermore, Lemma 2 in the source explicitly mentions that for any two distinct configurations $x_1, x_2 \in S_i \cup T_i$ (start and target positions within a component $F_i$), $D^*(x_1) \cap D^*(x_2) = \emptyset$, because $D_2(x_1) \cap D_2(x_2) = \emptyset$ due to the assumption that the start and target positions are well-separated. Here, $D^*(x) = D_2(x) \cap F_i$ is the part of the collision disc within the same free-space component as $x$.

    The paper notes in that while a separation distance of 4 ensures that a solution always exists (given that each connected component contains the same number of start and target positions), this might not be the case with a smaller separation distance, as illustrated in Figure 4. In the example, even with the same number of start and target positions in the free space, a smaller separation prevents the robots from simultaneously reaching their goals because one robot blocks the other’s path.

    K-Color Multi-Robot Motion Planning: An Overview

    Based on the sources, the k-color motion-planning problem is a generalization of the unlabeled multi-robot motion planning problem.

    In this variant, instead of having a single group of completely identical and interchangeable robots (as in the standard unlabeled problem), there are several distinct groups of interchangeable robots, where each group can be considered a different “color”.

    Therefore, the key characteristic of the k-color motion-planning problem is the presence of k different groups (colors) of robots, where robots within the same group are interchangeable, but robots from different groups are distinguishable (by their color) and are not interchangeable with robots from other groups.

    The source cites as the origin of this problem proposal.

    Hierarchical Navigation Control Algorithm for Multi-Particle Systems

    The core idea of the Hierarchical Navigation Control (HNC) Algorithm is to solve the centralized motion planning problem for multiple distinct Euclidean particles by using a hierarchical decomposition of the configuration space based on the HC2-means clustering method. It achieves this by combining continuous control within individual strata with discrete transitions between adjacent strata in a provably correct and computationally efficient manner.

    Here’s a breakdown of the key aspects of the HNC Algorithm:

    • Hierarchical Decomposition of Configuration Space: The algorithm utilizes the concept of strata, where each stratum $S(\tau)$ corresponds to a specific binary hierarchy $\tau$ of the particles obtained through HC2-means clustering. The entire configuration space is covered by these tree-indexed strata [as implied by previous conversations].
    • Intra-Stratum Navigation (Hybrid Base Case): When the current configuration $x$ belongs to the same stratum $S(\tau)$ as the desired goal configuration $y$ (which supports $\tau$, $y \in \stackrel{\circ}{S}(\tau)$), the algorithm applies a stratum-invariant continuous controller $f_{\tau,y}$ (from Algorithm 1 in). This controller, based on hierarchy-invariant vector fields, ensures that the system stays within $S(\tau)$ and asymptotically approaches the goal $y$ without collisions. This is treated as the “hybrid base case”.
    • Inter-Stratum Navigation (Hybrid Recursive Step): If the current configuration $x$ (supporting hierarchy $\sigma$) is not in the same stratum as the goal $y$ (supporting hierarchy $\tau$), the algorithm enters a “hybrid recursive step” to navigate across strata. This involves:
    • Discrete Transition in Hierarchy Space: Invoking the NNI (Nearest Neighbor Interchange) transition rule $g_{\tau}$ (from Algorithm 2 in) on the space of binary trees $BT_J$. This rule proposes an adjacent hierarchy $\gamma$ in the NNI-graph that is closer to the goal hierarchy $\tau$ in terms of a discrete Lyapunov function. The NNI-graph $N_J$ is a subgraph of the adjacency graph $A_J$ of the HC2-means hierarchies.
    • Defining a Local Goal using the Portal Map: Choosing a local configuration goal $z$ within the portal between the current stratum $S(\sigma)$ and the proposed adjacent stratum $S(\gamma)$. This local goal $z$ is computed using the portal map $Port(\sigma, \gamma)(x)$. The portal $Portal(\sigma, \tau) = \stackrel{\circ}{S}(\sigma) \cap \stackrel{\circ}{S}(\tau)$ represents the set of configurations supporting interior strata of both hierarchies. The portal map provides a computationally effective geometric realization of the edges of the NNI-graph in the configuration space. It retracts $S(\sigma)$ into the set of standard portal configurations in $Portal(\sigma, \gamma)$.
    • Continuous Control Towards the Local Goal: Applying another stratum-invariant continuous controller $f_{\sigma,z}$ (from Algorithm 1) to drive the system from the current configuration $x$ within $S(\sigma)$ towards the local goal $z \in Portal(\sigma, \gamma)$. This ensures the state remains within $S(\sigma)$ during this phase.
    • Transitioning to the Next Stratum: Once the trajectory reaches a sufficiently small neighborhood of $z$ (and hence enters $Portal(\sigma, \gamma) \subset S(\gamma)$ in finite time), the algorithm updates the current hierarchy to $\sigma \leftarrow \gamma$ and repeats the recursive step (2a) until the configuration enters the goal stratum $S(\tau)$, at which point the base case (step 1) is applied.
    • Hybrid Dynamical System: The HNC Algorithm defines a hybrid dynamical system by alternating between discrete transitions in the space of hierarchies (using the NNI-graph) and continuous motion within the strata (using hierarchy-invariant vector fields and the portal map).
    • Correctness and Efficiency: The algorithm guarantees that almost every initial configuration will reach an arbitrarily small neighborhood of the desired goal configuration $y$ in finite time, without any collisions along the way. Each discrete transition and the computation of the portal location can be done in linear time, $O(|J|)$, with respect to the number of particles $|J|$. The NNI transition rule $g_{\tau}$ ensures progress towards the goal hierarchy $\tau$ by reducing a discrete Lyapunov function.

    In summary, the HNC Algorithm’s core idea is to systematically navigate through the configuration space by moving within well-defined strata using continuous, collision-avoiding control, and transitioning between adjacent strata (that are closer in the hierarchical clustering space) using a discrete process guided by the NNI-graph and geometrically realized by the portal map. This hybrid approach provides a computationally effective and provably correct method for multi-particle motion planning.

    By Amjad Izhar
    Contact: amjad.izhar@gmail.com
    https://amjadizhar.blog

  • AngularJS in Action by Lukas Ruebbelke Using Framework, Philosophy Best Practices

    AngularJS in Action by Lukas Ruebbelke Using Framework, Philosophy Best Practices

    “AngularJS in Action” by Lukas Ruebbelke is a book aimed at equipping readers with practical techniques for building web applications using the AngularJS framework. It covers fundamental concepts like directives, controllers, services, and routing, progressing to more advanced topics such as animations, form validation, and server communication using $http. The book introduces the AngularJS philosophy of extending HTML for dynamic web applications and emphasizes best practices for structuring and testing AngularJS projects through examples like the Angello application, a Trello clone. Additionally, the text mentions supplementary online resources and contributions from the AngularJS community and reviewers.

    AngularJS Study Guide

    Quiz

    1. Explain the primary purpose of AngularJS modules. How does using sub-modules contribute to better application design?
    2. Describe the role of controllers in AngularJS applications. How does the “controller as” syntax differ from traditional controller declaration, and what are its benefits?
    3. What are AngularJS directives, and why are they considered a powerful feature? Provide an example of a built-in directive and a scenario where a custom directive would be beneficial.
    4. Explain the AngularJS digest cycle. Why is it important, and when might you need to manually trigger it using $apply?
    5. What are AngularJS services, and what is their main purpose in an application? Briefly describe two different types of services discussed in the source material.
    6. Describe how AngularJS handles client-side routing using ngRoute. What are the key components involved in defining and using routes?
    7. Explain the concept of isolated scope in AngularJS directives. What are the three types of isolated scope, and when would you use each?
    8. How does AngularJS facilitate form validation? Describe at least two built-in validation directives and how CSS classes are used to reflect the validation state.
    9. Outline the basic steps involved in testing AngularJS components like services and directives using tools like Karma and Jasmine.
    10. Describe how AngularJS handles animations. Briefly explain the different approaches discussed in the source material for implementing animations.

    Quiz Answer Key

    1. AngularJS modules serve as containers for organizing an application into logical units, defining how the application is configured and behaves. Using sub-modules to divide features makes the codebase more modular, easier to maintain, test, and move around independently.
    2. Controllers in AngularJS are responsible for managing the application’s data and behavior within a specific scope, acting as an intermediary between the view and the model. The “controller as” syntax assigns the controller instance to a scope variable, allowing direct access to its properties and methods in the view using that alias (e.g., {{storyboard.someProperty}}), which improves code clarity and avoids potential naming conflicts.
    3. AngularJS directives are markers on DOM elements that instruct AngularJS to attach specific behavior or transform the element and its children. They enhance HTML by creating reusable components and domain-specific language. ng-click is a built-in directive, while a custom directive could be used to encapsulate a reusable UI component like a custom date picker.
    4. The AngularJS digest cycle is the process of comparing the current scope values with their previous values to detect any changes and update the DOM accordingly. It’s crucial for two-way data binding. You might need to manually trigger it using $apply when changes occur outside of Angular’s awareness, such as within a callback function of a third-party library.
    5. AngularJS services are singleton objects that carry out specific tasks or provide reusable functionality across an application, promoting separation of concerns. Two types of services are value services, which are simple values or objects registered with a name, and service services, which are instantiated via a constructor function, allowing for more complex logic and the use of this for defining methods and properties.
    6. AngularJS routing with ngRoute involves defining URL patterns (routes) and associating them with specific views (templates), controllers, and potentially data loading logic (resolve). Key components include $routeProvider for defining routes, ng-view directive as a placeholder for the current view, and $location service for interacting with the browser’s URL.
    7. Isolated scope in AngularJS directives prevents the directive’s scope from prototypically inheriting from its parent scope, creating a distinct scope. The three types are: attribute-isolated scope (@) for one-way (parent to child) string binding, binding-isolated scope (=) for two-way binding of values, objects, or collections, and expression-isolated scope (&) for allowing the child scope to execute an expression in the parent scope. Isolated scope is used to create reusable and encapsulated components, avoiding unintended side effects.
    8. AngularJS offers built-in directives like ng-required to ensure a field is filled, and ng-minlength/ng-maxlength to enforce length constraints. AngularJS automatically adds CSS classes like .ng-valid, .ng-invalid, .ng-dirty, and .ng-pristine to form elements based on their validation state and user interaction, allowing developers to style them accordingly.
    9. Testing AngularJS services typically involves using angular-mocks to inject the service and its dependencies into a test suite (using Jasmine, for example). You can then call the service’s methods and use assertions to verify the expected behavior. Testing directives involves compiling an HTML element containing the directive using $compile and a $rootScope, then accessing the directive’s scope or controller to perform assertions on its behavior and DOM manipulation.
    10. AngularJS handles animations by leveraging the ngAnimate module, which detects structural changes in the DOM (like elements entering or leaving via ngRepeat, ngIf, etc.) and applies CSS transitions, CSS animations, or JavaScript animations. CSS transitions define property changes over a duration, CSS animations use @keyframes to define animation sequences, and JavaScript animations allow for more complex programmatic control using JavaScript code, often with libraries like TweenMax.

    Essay Format Questions

    1. Discuss the benefits of using a modular structure in large-scale AngularJS applications. Explain how modules, sub-modules, and services contribute to maintainability, testability, and overall organization, referencing specific examples from the source material.
    2. Analyze the role of directives in extending HTML and creating reusable UI components in AngularJS. Compare and contrast the different ways directives can be defined and used (e.g., element, attribute), and discuss the significance of scope in the context of directive development.
    3. Evaluate the Model-View-ViewModel (MVVM) pattern as it is implemented in AngularJS, focusing on the interactions between scopes, controllers, and views. Discuss the advantages and potential drawbacks of AngularJS’s approach to data binding and the digest cycle.
    4. Examine the importance of client-side routing in single-page applications (SPAs) built with AngularJS. Discuss the features provided by ngRoute, including route definition, parameter handling, and the use of resolve, and consider its limitations as mentioned in the source.
    5. Discuss the various techniques for enhancing the user experience of AngularJS applications through animations. Compare and contrast CSS transitions, CSS animations, and JavaScript animations, highlighting their strengths, weaknesses, and appropriate use cases based on the source material.

    Glossary of Key Terms

    • Module: A container in AngularJS that organizes different parts of an application, such as controllers, services, and directives, into logical units.
    • Controller: A JavaScript function associated with a specific scope and view, responsible for providing data to the view and defining the application’s behavior.
    • View: The HTML template rendered by AngularJS, which displays data from the scope and allows user interaction.
    • Scope: An object that refers to the application model and acts as a context for evaluating expressions. It serves as the glue between the controller and the view.
    • Directive: A marker on a DOM element that tells AngularJS to attach a specific behavior or transform the element and its children.
    • Service: A singleton object that encapsulates reusable business logic or utility functions, making them available across the application.
    • Factory: A type of service defined by a function that creates and returns a value or an object (the service instance).
    • Provider: The most configurable type of service definition, allowing access to the service’s configuration during the application’s configuration phase.
    • $http: A core AngularJS service that facilitates communication with remote HTTP servers to perform CRUD operations.
    • Promise: An object representing the eventual completion (or failure) of an asynchronous operation and its resulting value.
    • Digest Cycle: The internal process in AngularJS that checks for changes in the scope variables and updates the DOM accordingly.
    • ngRoute: An AngularJS module that provides client-side routing capabilities for single-page applications.
    • $routeProvider: A service provided by ngRoute used to define URL routes, associate them with templates and controllers, and configure route-specific settings.
    • ngView: A directive provided by ngRoute that acts as a placeholder in the main HTML template where the content of the current route will be rendered.
    • Isolated Scope: A scope in a directive that does not prototypically inherit from its parent scope, providing encapsulation and preventing unintended side effects.
    • ngAnimate: An AngularJS module that enables support for CSS-based and JavaScript-based animations.
    • CSS Transition: A way to smoothly change CSS property values over a specified duration.
    • CSS Animation: A way to define keyframe-based animations using CSS rules.
    • Karma: A JavaScript test runner that allows you to execute tests in real browsers.
    • Jasmine: A behavior-driven development (BDD) framework for testing JavaScript code.

    Briefing Document: AngularJS in Action Excerpts

    This briefing document summarizes key themes and important ideas from the provided excerpts of “AngularJS in Action.” The book appears to be a practical guide to developing applications using the AngularJS framework (version 1.x, as indicated by the version of angular-animate.min.js included).

    Part 1: Getting Acquainted with AngularJS

    Chapter 1: Hello AngularJS

    Main Themes:

    • AngularJS as a Comprehensive Framework: The book positions AngularJS as a solution for building dynamic web applications, emphasizing its ability to organize and structure code.
    • “If you reach the end of the book and you have a solid grasp of figure 1.1 and how all the pieces fit together, we’ll have succeeded as authors.” (p. 7)
    • Modularity: AngularJS utilizes modules as containers for organizing an application into logical units. This promotes maintainability and testability.
    • “Modules in AngularJS serve as containers to help you organize your application into logical units. Modules tell AngularJS how an application is configured and how it’s supposed to behave.” (p. 10) “It’s considered best practice to divide features into sub-modules and then inject them into the main application module. This makes it much easier to move a module around as well as test it.” (p. 10)
    • Key Components: The chapter introduces fundamental AngularJS components like modules, services, controllers, and directives, highlighting their roles in the application’s architecture (illustrated in figures 1.5 and 1.9).
    • Directives as Powerful Tools: Directives are presented as a core and powerful feature for augmenting HTML behavior and creating reusable components.
    • “Directives are one of the most powerful and exciting things in AngularJS.” (p. 17) “When you add ng-app or ng-controller to the page, you’re using AngularJS directives to provide new behavior to an otherwise static page.” (p. 17)
    • Initial Application Setup: The chapter provides a basic example of creating a module (Angello) and defining simple services, controllers, and a directive.

    Chapter 2: Structuring Your AngularJS Application

    Main Themes:

    • Modular Application Design: This chapter reinforces the importance of structuring an application using modules, demonstrating how a larger application (Angello) is broken down into feature-specific sub-modules (e.g., Angello.Common, Angello.Dashboard, Angello.Login).
    • “We also have a sub-module for every feature of Angello, including one for the common functionality that’s shared between the features. This allows us to look at how the Angello module is being constructed and quickly establish a mental picture of the pieces that make up the application.” (p. 25)
    • Separation of Concerns: The structure emphasizes separating different functionalities into distinct modules.
    • Importance of Correct Module Retrieval: A caution is given regarding the angular.module() syntax, emphasizing that calling it with an empty array as the second parameter will overwrite an existing module definition.
    • “PLEASE BE CAREFUL To get an AngularJS module, you’ll call angular.module without the second parameter. We’ve unfortunately run into some unpredict-able behavior by accidentally putting in an empty array as the second parame-ter, which will overwrite the module definition and create a new one.” (p. 26)
    • Routing with ngRoute: The chapter introduces the ngRoute module for handling client-side routing and navigation. It shows how to define routes and associate them with templates and controllers.
    • “We’ll spend the rest of this chapter dis-cussing the various parts that make routes possible in AngularJS, while showing how we can use it in Angello.” (p. 130) (Note: This quote is from Chapter 7 but its foundation is laid here with the inclusion of ngRoute).
    • ngView Directive: The role of the ngView directive as a placeholder for the rendered template of the current route is explained.
    • “We accomplish this by adding <div ng-view=””></div> into our main layout file. ngView is responsible for fetching the route template and compiling it with the route’s controller and displaying the finished, compiled view to the user.” (p. 132)
    • Basic Route Configuration: Examples demonstrate how to use $routeProvider to define routes with templateUrl, controller, and controllerAs properties.
    • Default Route: The use of .otherwise() to redirect to a default route is shown.

    Part 2: Making Something with AngularJS

    Chapter 3: Views and Controllers

    Main Themes:

    • Views and Controllers Interaction: This chapter focuses on how views (HTML templates) and controllers (JavaScript functions) work together in AngularJS. The $scope is highlighted as the “glue” between them.
    • “Scope: Glue Controller: Imperative behavior View (DOM): Declarative view Figure 3.3 Scope is the glue” (p. 37)
    • MVVM Pattern: AngularJS is described as adhering to the Model-View-ViewModel (MVVM) pattern.
    • “Model-View-ViewModel (MVVM) According to AngularJS Figure 3.4 MVVM according to AngularJS” (p. 38)
    • The Digest Cycle and Dirty Checking: The concept of the digest cycle and dirty checking is introduced as the mechanism by which AngularJS detects and propagates changes in the model to the view.
    • “Dirty checking is the simple process of comparing a value with its previous value, and if it has changed, then a change event is fired.” (p. 40) “AngularJS performs dirty checking via a digest cycle that’s controlled by $digest. $digest happens implicitly, and you never have to call this directly. If you need to ini-tiate a digest cycle, then use $apply; it calls $digest but has error-handling mecha-nisms built around it.” (p. 40)
    • Controller as Syntax: The controllerAs syntax (introduced in AngularJS 1.3) is explained as a way to refer to the controller instance directly in the view, improving clarity.
    • “In AngularJS 1.3, a new convention was introduced for working with controllers known as the controller-as syntax. In a hypothetical situation, instead of declaring a controller on the view as ng-controller=”StoryboardCtrl”, you’d define it as ng-controller =”StoryboardCtrl as storyboard”. Throughout the rest of the view, you’d refer to that controller as storyboard.” (p. 41)
    • AngularJS Events: The $broadcast and $emit events for communication up and down the scope hierarchy are mentioned.
    • Properties and Expressions: The use of expressions ({{ }}) in views for data binding is covered.
    • ngRepeat Directive: The ngRepeat directive for iterating over collections and displaying data is demonstrated, along with its implicit creation of child scopes.
    • Filters: Filters for formatting and transforming data within expressions and ngRepeat are introduced.
    • One-Time Data Binding: AngularJS 1.3’s one-time binding syntax (::) for performance optimization is highlighted.

    Chapter 4: Models and Services

    Main Themes:

    • Models and Services Defined: This chapter explains the roles of models (for data) and services (for business logic and reusable functionality).
    • “What are models and services?” (p. 58)
    • Service Registration: Different ways to register services with an AngularJS module are introduced: value, constant, service, factory, and provider.
    • “Services are ultimately registered with the application with the built-in $provide service, but in most cases it’s easier to use the syntactic sugar provided by angular.module. These convenience methods are module.value, module.constant, module.service, module.factory, and module.provider.” (p. 59)
    • Service Lifecycle: The instantiation and caching of services by the $injector are described.
    • Different Service Types: The characteristics and use cases of value, constant, and service types are detailed with examples. The service type is shown to be useful for object-oriented approaches using the this keyword.
    • Models with $http: The $http service for making HTTP requests to the server is introduced as a common way to fetch and persist data (models). Convenience methods like .success() and .error() are mentioned.
    • Promises ($q): Promises are explained as a way to handle asynchronous operations, providing a more elegant way to manage callbacks with .then(), .catch(), and .finally().
    • “What are promises?” (p. 68)
    • $http Interceptors: The concept of interceptors for pre-processing requests and post-processing responses of $http calls is introduced, along with use cases like logging and authentication.
    • “Why intercept?” (p. 71)
    • Service Decorators: Decorators, using the $provide.decorator() method, are explained as a way to enhance or modify existing services. An example of enhancing $log with timestamps is provided.
    • “Why decorate?” (p. 73)
    • Testing Services: Strategies for testing AngularJS services are covered, including using beforeEach(module(…)) and beforeEach(inject(…)). The underscore wrapping convention for injected dependencies is noted.
    • “Note that we actually inject $rootScope and LoadingService as parameters. This is called underscore wrapping and is done so that we can assign those variables to the actual service name in our code. The inject method knows to strip out the under-scores and return the actual service.” (p. 76)
    • $httpBackend for Mocking: The $httpBackend service is introduced as a tool for mocking server-side interactions during testing.

    Chapter 5: Directives

    Main Themes:

    • Introduction to Directives: Directives are presented as a core mechanism for extending HTML, creating reusable UI components, and building domain-specific languages (DSLs).
    • “One outstanding feature of directives is that they allow you to turn your HTML into a domain-specific language.” (p. 81)
    • Directive Definition Object: The structure of a directive definition object, including properties like restrict, link, controller, and controllerAs, is explained. The restrict property (e.g., ‘A’ for attribute) is highlighted.
    • link and controller Functions: The roles of the link function (for DOM manipulation and event binding) and the controller function (for adding behavior to the directive’s scope) are described.
    • Scope in Directives: The use of scope: true for creating an isolate scope is mentioned.
    • Building Advanced Directives: The chapter walks through the creation of complex directives like a drag-and-drop feature, demonstrating the interaction of multiple directives and a service ($dragging).
    • Integrating with Third-Party Libraries: An example of integrating with the Flot charting library using a directive is provided. This involves installing the library, building a directive to interact with it, and massaging data into the expected format.
    • Isolated Scope in Detail: The concept of isolated scope (scope: { … }) is discussed in detail, including attribute-isolated scope (@), binding-isolated scope (=), and expression-isolated scope (&).
    • “AngularJS allows you to accomplish this via isolated scope, which creates an ironclad perimeter around the directive’s scope, and then it’s the responsibility of the developer to define exactly how the directive will communicate with the outside world. This essen-tially provides an API for your directive with clearly defined channels of communication.” (p. 109)
    • Testing Directives: The process of testing directives involves creating an Angular element, compiling it with $rootScope, and then accessing its scope and controller.
    • Best Practices for Directives: Favoring a compartmentalized approach and breaking down large directives into smaller, reusable components is recommended.

    Chapter 6: Animations

    Main Themes:

    • Introduction to AngularJS Animations (ngAnimate): This chapter introduces the ngAnimate module for adding animations to AngularJS applications.
    • “Now that angular-animate.min.js has been included, we need to inject it as a sub-module into our application:” (p. 117)
    • Animation Naming Convention: The naming convention for CSS classes used by ngAnimate (e.g., .my-animation.ng-enter, .my-animation.ng-leave) is explained.
    • Enabling Animations: The steps to enable animations (including angular-animate.min.js and injecting ‘ngAnimate’ as a module dependency) are outlined.
    • CSS Transitions: How to define CSS transitions for elements entering (ng-enter) and leaving (ng-leave) the DOM is demonstrated using CSS rules.
    • CSS Animations: The use of CSS @keyframes to define more complex animations triggered by ng-enter, ng-leave, and ng-move events is shown, along with the need for vendor prefixes.
    • JavaScript Animations: Implementing animations using JavaScript by registering animation hooks with the .animation() method on a module is explained. The events (addClass, removeClass) and the done callback are highlighted.
    • TweenMax: The TweenMax library (from GreenSock) is introduced as a tool for creating more sophisticated JavaScript-based animations.
    • Testing Animations: Basic considerations for testing animations are mentioned.

    Chapter 7: Structuring Your Site with Routes

    Main Themes:

    • AngularJS Routing with ngRoute: This chapter delves deeper into using the ngRoute module to manage application states based on the URL.
    • “Routes help you intelligently decide what to show and how to show it, based on the URL of the application.” (p. 130)
    • Components of AngularJS Routes: The key components of routing (e.g., $routeProvider, $route, $routeParams, ngView) are outlined.
    • Creating Routes: The use of $routeProvider.when() to define routes with templateUrl, controller, and controllerAs is demonstrated.
    • ngView Limitation: The limitation of ngRoute allowing only one ngView directive per page is noted, and AngularUI Router is suggested as an alternative for complex layouts.
    • “NGROUTE LIMITATION You’re only allowed to declare one ng-view on your page; this is one of the most glaring shortcomings of ngRoute.” (p. 27)
    • Route Parameters: How to define and access route parameters (e.g., /users/:userId) using $routeParams is explained.
    • resolve Property: The resolve property in route definitions is introduced as a way to fetch dependencies (e.g., user data) before the controller is instantiated.
    • Route Events: Events like $routeChangeStart, $routeChangeSuccess, and $routeChangeError that are broadcast during the routing process are mentioned.
    • Testing Routes: Basic approaches to testing route configurations are shown, involving using $location.path() and $rootScope.$digest() to simulate navigation and then asserting the properties of the current route.

    Chapter 8: Forms and Validations

    Main Themes:

    • AngularJS Form Validation: This chapter covers AngularJS’s built-in form validation features.
    • “The big picture: AngularJS form validation” (p. 143)
    • Extending HTML Form Elements: AngularJS extends HTML form elements with directives and properties for validation.
    • Adding Validations: Built-in validation directives like required, ng-minlength, and ng-maxlength are demonstrated.
    • Form Validation and CSS: How AngularJS adds CSS classes (.ng-valid, .ng-invalid, .ng-dirty, .ng-pristine) to form elements based on their validation state, allowing for visual feedback, is explained.
    • $setPristine and $setUntouched: These form methods for resetting the form’s state are mentioned.
    • Custom and Asynchronous Validation: The possibility of creating custom validation directives using ngModel and $validators is introduced.
    • Testing Forms: Basic strategies for testing form validation logic are covered.
    • Best Practices for Forms: Recommendations for structuring forms, such as using nested forms, are provided.

    Appendices

    • Appendix A: Setting up Karma: Provides instructions on setting up the Karma test runner, including installing Node.js, npm, and Karma-related packages, and configuring karma.conf.js.
    • Appendix B: Setting up a Node.js Server: Offers guidance on setting up a basic Node.js server for the application’s backend.
    • Appendix C: Setting up a Firebase Server: Provides information on using Firebase as a backend.
    • Appendix D: Running the App: Explains the steps to run the AngularJS application, including cloning the repository and starting a local web server (if needed). It also touches on configuring authentication with Auth0.

    Index

    The index provides a comprehensive list of terms and concepts covered in the book, facilitating quick lookups. It includes entries for various AngularJS components (e.g., directives, controllers, services), modules (ngAnimate, ngRoute), concepts (e.g., data binding, routing, validation), and third-party libraries (e.g., Flot, TweenMax).

    Overall, the provided excerpts demonstrate a practical, hands-on approach to learning AngularJS 1.x development. The book covers fundamental concepts like modules, directives, controllers, services, routing, forms, and animations, illustrating them with code examples and best practices. It also emphasizes testing as an integral part of the development process and explores integration with external libraries.

    FAQ: AngularJS Development with Insights from “AngularJS in Action”

    • What is the core purpose of AngularJS modules? AngularJS modules serve as organizational containers for different parts of your application, such as controllers, services, directives, and configuration. They help in structuring the application into logical and manageable units. By dividing features into sub-modules and injecting them into the main application module, it becomes easier to maintain, test, and reuse components across different parts of the application. The angular.module() function is used to define or retrieve existing modules.
    • How do AngularJS views and controllers work together? In AngularJS, views are the HTML templates that define the structure and presentation of the user interface, while controllers are JavaScript functions that provide the behavior and data to the views. Controllers are attached to the view using directives like ng-controller. They interact with the $scope object, which acts as the glue between the view and the controller, allowing for data binding. Changes in the controller’s data are automatically reflected in the view, and vice versa, through AngularJS’s data binding mechanisms. The “controller as” syntax provides an alternative way to reference the controller instance directly within the view, making the code more explicit.
    • What are AngularJS services and what are some common types? AngularJS services are singleton objects that carry out specific tasks within an application. They are used to organize and share code across different components like controllers, directives, and other services. Services promote code reusability and maintainability. Common types of services include:
    • Value services: Simple services that return a specific value (primitive, object, or function).
    • Constant services: Similar to value services but cannot be modified during the application’s lifecycle and can be injected into the config block.
    • Service factories: Services created using a constructor function, ideal for object-oriented approaches where methods and properties are defined on this.
    • Factory services: Services created by a function that returns the service object.
    • Provider services: The most configurable type, allowing for configuration during the application’s configuration phase before the service is instantiated.
    • How does AngularJS handle asynchronous operations using $http and promises? The $http service in AngularJS is used to make HTTP requests to the server to fetch or send data. It returns promise objects that represent the eventual outcome (success or failure) of the asynchronous operation. Promises provide a structured way to handle the results of these operations using methods like .then() for success callbacks, .catch() for error callbacks, and .finally() which executes regardless of the outcome. AngularJS also provides $http convenience methods like $http.get(), $http.post(), etc., for common HTTP methods. $http interceptors can be used to preprocess or postprocess HTTP requests and responses, allowing for tasks like authentication, logging, or error handling.
    • What are AngularJS directives and why are they important? Directives are powerful features in AngularJS that allow you to extend HTML with new attributes and elements, essentially turning your HTML into a domain-specific language. They encapsulate specific behaviors and DOM manipulations, making your markup more expressive and reusable. Directives can be used to augment the behavior of existing HTML elements, create reusable UI components, and manipulate the DOM in a controlled way. They are defined using the angular.module().directive() method and involve concepts like scope, templates, and link functions to define their behavior.
    • How does AngularJS handle animations? AngularJS provides the ngAnimate module to easily add animations to your application. It works by hooking into AngularJS’s lifecycle events for directives like ng-repeat, ng-if, ng-show, and ng-hide, and applying CSS classes (e.g., .ng-enter, .ng-leave) at different stages of the animation. You can define animations using CSS transitions, CSS animations (with keyframes), or JavaScript animations. For JavaScript animations, you can register animation hooks using the .animation() function on a module, allowing you to use libraries like TweenMax for more complex animations.
    • What is AngularJS routing and how is it implemented using ngRoute? AngularJS routing, typically implemented using the ngRoute module, allows you to map application states to specific URLs. This enables you to build single-page applications where navigating to different URLs loads different views and executes corresponding controllers without a full page reload. The $routeProvider service is used within the config block of a module to define routes using the .when() method, specifying the URL path, the template to be loaded (templateUrl), and the controller to be associated with that view (controller and controllerAs). The ng-view directive is placed in the main HTML layout to indicate where the content of the current route should be rendered. Route parameters can be defined in the URL (e.g., /users/:userId) to pass dynamic data to the controller, and the resolve property can be used to load dependencies before the route is activated.
    • How does AngularJS facilitate form handling and validation? AngularJS provides built-in support for form handling and validation. When you use the <form> tag and input elements with ng-model, AngularJS automatically tracks the state of the form and its controls. It adds various properties to the form and input elements’ $scope objects (e.g., $valid, $invalid, $dirty, $pristine, $touched, $untouched, $error) that you can use to display validation messages and style the form based on its state. AngularJS also includes built-in validation directives like required, ng-minlength, ng-maxlength, and ng-pattern. You can also create custom validation directives by interacting with the ngModelController’s $validators pipeline. Form validation status can be reflected in the UI using CSS classes (e.g., .ng-valid, .ng-invalid, .ng-dirty) that AngularJS automatically adds to form elements.

    AngularJS: An Overview of Core Concepts and Architecture

    AngularJS is an open-source web application framework that is described as “HTML enhanced for web apps!“. It allows developers to create dynamic, interactive single-page web interfaces in a manner similar to building standard static pages. AngularJS extends HTML to handle dynamic content, interactions, and animations. According to the foreword, it’s quickly becoming one of the front-end frameworks to use.

    The book highlights several advantages of using AngularJS:

    • It offers an intuitive framework that makes it easy to organize code in a way that promotes maintenance, collaboration, readability, and extension. It provides a structure where code related to UI behavior, the domain model, and DOM manipulation each have a logical place.
    • AngularJS was written from the ground up to be testable, making it easier to write clean, stable code that can scale and reducing the worry of unexpected application failures.
    • Two-way data binding saves developers from writing hundreds of lines of code by automatically synchronizing JavaScript properties with HTML bindings, eliminating the need for manual DOM manipulation for these tasks.
    • Because AngularJS templates are just HTML, it’s easy to leverage existing HTML skills for UI development.
    • The ability to work with Plain Old JavaScript Objects (POJOs) makes integration with other technologies incredibly easy for both consuming and emitting data.

    The “AngularJS big picture” consists of several key components:

    • Modules serve as containers for organizing code within an AngularJS application and can contain sub-modules.
    • Config blocks allow for configuration to be applied before the application runs, useful for setting up routes and configuring services.
    • Routes define ways to navigate to specific states within the application and configure options like templates and controllers.
    • Views are the result after AngularJS has compiled and rendered the DOM with JavaScript wiring in place. The compilation process involves a compilation phase (parsing DOM for directives) and a linking phase (linking directives to scope).
    • $scope acts as the glue between the view and the controller, facilitating the exposure of methods and properties for the view to bind to. The introduction of the “controller as” syntax has reduced the explicit need for $scope.
    • Controllers are responsible for defining methods and properties that the view can bind to and interact with. They should be lightweight and focused on the view they control. AngularJS encourages the separation of declarative markup (view) from imperative behavior (controller).
    • Directives are an extension of views, allowing for the creation of custom, reusable HTML elements that encapsulate behavior. They can be thought of as components or decorators for HTML. Directives typically have a Directive Definition Object (DDO), a link function for DOM manipulation, and a controller function for behavior.
    • Services provide common functionality that can be shared across the entire AngularJS application, extending controllers and making them more globally accessible. AngularJS offers different types of services like value, constant, service, factory, and provider.

    AngularJS is described as following a Model-View-Whatever (MVW) framework, which for the sake of conversation is often assumed to be the Model-View-ViewModel (MVVM) design pattern. In this context:

    • The View is the HTML rendered by AngularJS.
    • The Controller plays the role of the ViewModel, providing the state and commands for the view.
    • Services often represent the Model, handling data and business logic, including communication with remote servers (in which case they might be referred to as models within the book’s context). The $http service is built-in for server-side communication using XMLHttpRequest or JSONP and employs a promise-based API via the $q service.

    The book uses a sample application called Angello, a Trello clone for managing user stories, to illustrate various AngularJS concepts. A simplified version, Angello Lite, is built in the initial chapters to introduce the core components.

    AngularJS is designed to be testable from the ground up. The book covers testing various components like controllers, services, and directives. Tools like Karma, a JavaScript test runner created by the AngularJS team, are discussed for setting up the testing environment.

    AngularJS: A Framework for Web Application Development

    Web application development, as discussed within the context of the provided source, has evolved significantly. Initially, any logic on a web page required server-side processing and a full page reload, leading to a disjointed user experience. The introduction of XMLHttpRequest enabled asynchronous communication, allowing for more interactive user experiences without constant page refreshes. This shift gave rise to the first wave of JavaScript frameworks.

    However, as web pages aimed to behave more like actual applications, new challenges emerged, particularly in organizing and maintaining large JavaScript codebases. While libraries like jQuery excelled at DOM manipulation, they lacked guidance on application structure, often resulting in unmanageable code.

    AngularJS emerged to address these challenges by providing a structured framework for building large, maintainable web applications. It offers several key advantages that streamline the development process:

    • Intuitive Code Organization: AngularJS provides a structure with clear places for different types of code, such as UI behavior (controllers), data (services/models), and DOM manipulation (directives). This makes it easier to maintain, collaborate on, read, and extend applications. The source code for the Angello application is organized by feature, demonstrating a practical approach to structuring an AngularJS project.
    • Testability: AngularJS was designed with testability as a core principle, making it easier to write unit and integration tests to ensure the application works correctly and remains stable as it evolves. The book includes appendix A on setting up Karma, a JavaScript test runner, and discusses testing various AngularJS components.
    • Two-Way Data Binding: This feature automatically synchronizes data between the JavaScript model and the HTML view, reducing the amount of boilerplate code needed to keep them in sync.
    • HTML Templates: AngularJS uses standard HTML as templates, making it easy for developers and UI/UX designers familiar with HTML to contribute to the front-end. AngularJS enhances HTML with directives to overcome its inherent limitations for building complex interactions.
    • Easy Integration with JavaScript Objects: AngularJS works with Plain Old JavaScript Objects (POJOs), simplifying integration with other technologies and data sources.

    AngularJS follows a Model-View-Whatever (MVW) architecture, often conceptualized as Model-View-ViewModel (MVVM). In this pattern:

    • The View is the HTML template rendered by AngularJS.
    • The Controller (or ViewModel) manages the data and behavior exposed to the view. The “controller as” syntax further clarifies the controller’s role in the view.
    • Models (often implemented as services in AngularJS) handle the application’s data and business logic, including communication with back-end servers using services like $http.

    Directives are a key aspect of AngularJS that extend HTML by creating custom tags and attributes with specific behaviors, enabling the development of reusable UI components.

    Routing is another crucial element for single-page applications, allowing navigation between different views based on the URL. AngularJS provides the ngRoute module (and the book recommends exploring the more advanced ui-router) to manage application states based on URL changes.

    Finally, forms and validations are essential for web applications to handle user input and ensure data integrity. AngularJS extends HTML forms, providing mechanisms for data binding, validation rules (like ng-required, ng-minlength, ng-maxlength), and feedback to the user.

    The development of Angello throughout the book serves as a practical example of building a web application using these AngularJS principles. The initial simplified version, Angello Lite, helps to introduce the core concepts. The book also touches upon integrating with back-end services like Firebase and Node.js, as well as third-party libraries like Flot for creating charts.

    In summary, AngularJS provides a comprehensive framework for web application development by offering structure, promoting testability, simplifying data binding, leveraging existing HTML skills, and facilitating integration with other technologies. It addresses the challenges of building modern, interactive web applications by providing tools and patterns for organizing code, managing UI interactions, handling data, and ensuring a maintainable and scalable codebase.

    AngularJS Directives: Extending HTML for Components

    Directives are a fundamental concept in AngularJS, serving as a powerful mechanism to extend HTML with custom behavior and create reusable UI components. The book emphasizes that AngularJS was designed to enhance HTML for web applications, and directives are the primary way this is achieved.

    Here’s a breakdown of directives based on the provided sources:

    • Definition and Purpose:
    • Directives are custom HTML tags and attributes that you can create to add new behaviors to the DOM.
    • They allow you to overcome the limitations of static HTML when building dynamic and interactive web applications.
    • By using directives, you can write more expressive and domain-specific HTML, effectively turning your HTML into a domain-specific language (DSL). For example, the book discusses creating a <story> tag using a directive to represent a user story.
    • Structure of a Directive:
    • A directive generally consists of three main parts:
    • Directive Definition Object (DDO): This object defines how the directive should be configured and behave during the AngularJS compilation cycle. It includes properties such as:
    • restrict: Specifies how the directive can be used in the HTML (e.g., as an attribute ‘A’, element ‘E’, class ‘C’, or comment ‘M’). The userstory directive in the book is restricted to be used as an attribute (restrict: ‘A’).
    • template or templateUrl: Defines the HTML markup that the directive will render.
    • replace: Determines whether the directive’s element is replaced by the template.
    • scope: Configures the scope that the directive will use (e.g., prototypical inheritance or isolated scope).
    • link: A function that allows you to perform DOM manipulation and add event listeners. It receives the scope, the element (wrapped in jQuery), and the element’s attrs as parameters.
    • controller and controllerAs: Define the controller function associated with the directive and how it can be referenced in the template.
    • require: Specifies dependencies on other directives’ controllers.
    • Link Function: This function is primarily used for DOM manipulation, event handling, and interaction with third-party libraries. The book provides an example of adding a fade effect on mouseover using the link function of a userstory directive.
    • Controller Function: This function is responsible for defining the behavior and data that the directive’s template can bind to. Services can be injected into the directive’s controller.
    • Types of Directives and Scope:
    • Directives can have different types of scope:
    • Prototypical Scope (default): The directive’s scope inherits prototypically from its parent scope.
    • Isolated Scope: This creates a completely new scope for the directive, providing better encapsulation and reusability. Communication with the parent scope needs to be explicitly defined through the scope property in the DDO, using prefixes like @ (attribute-isolated, one-way, string value), = (binding-isolated, two-way), and & (expression-isolated, allows executing parent expressions). The chart directive in the book uses binding-isolated scope to receive data from the parent controller.
    • Directives as Components:
    • The book explicitly states, “You can think of directives as components or decorators for your HTML“.
    • Directives, especially those with isolated scope, a defined template, and a controller, strongly embody the concept of reusable UI components. They encapsulate both the structure (template), behavior (controller), and styling (often through associated CSS classes) of a specific UI element or functionality.
    • The drag-and-drop feature built with drag-container, drop-container, and drop-target directives illustrates how multiple directives can work together to create a complex component. Each directive focuses on a specific aspect of the drag-and-drop functionality, demonstrating a compartmentalized approach to building components.
    • Integration with Other AngularJS Features:
    • Directives interact closely with other parts of AngularJS, such as controllers (for providing data and behavior), services (for accessing and manipulating data or shared functionality), and scope (for data binding and communication).
    • They can also integrate with animations using the ngAnimate module by responding to specific CSS classes added and removed by AngularJS during DOM manipulations (like ng-enter, ng-leave).
    • Best Practices for Directives:
    • DOM manipulation should primarily be done in the link function, keeping the controller focused on business logic.
    • Favor a compartmentalized approach by breaking down complex functionality into smaller, independent, and reusable directives. This leads to cleaner and more maintainable code.

    In summary, directives are a cornerstone of AngularJS, enabling the creation of dynamic and reusable UI elements that function as components. They extend the capabilities of HTML, allowing developers to build complex single-page applications with a structured and maintainable approach. The concept of isolated scope further enhances the component-like nature of directives by promoting encapsulation and clear communication interfaces.

    AngularJS Architecture: Model-View-ViewModel Pattern

    The Model-View-ViewModel (MVVM) is an architectural pattern that AngularJS often follows to structure web applications, promoting a separation of concerns between the data (Model), the user interface (View), and the logic that manages the data for the view (ViewModel, which is typically the Controller in AngularJS). Our conversation history also mentioned MVVM as the pattern AngularJS often follows.

    Here’s a breakdown of how MVVM is realized in AngularJS based on the sources:

    • Model:
    • The Model is responsible for managing the application’s data and business logic.
    • In AngularJS, the Model is often implemented through services. Services can encapsulate data, communicate with remote servers using the $http service to persist data outside the client, and manage the state surrounding that data.
    • The AngelloModel service in the Angello Lite example and StoriesModel in the more comprehensive Angello application serve as examples of how models are implemented as services in AngularJS.
    • The Model notifies the ViewModel when the domain model has changed, so the ViewModel can update itself accordingly.
    • View:
    • The View is the HTML template that the user sees.
    • In AngularJS, the View is dynamically rendered and updated based on the data exposed by the ViewModel.
    • The View contains declarative markup and uses AngularJS directives (like ngRepeat, ngModel, ngClick) to bind to data and trigger actions in the ViewModel.
    • The View delegates responsibility by calling methods on the ViewModel.
    • ViewModel (Controller in AngularJS):
    • The ViewModel acts as the glue between the Model and the View. In AngularJS, the Controller largely fulfills the role of the ViewModel.
    • The Controller is a JavaScript object responsible for defining methods and properties that the View can bind to and interact with. This is often achieved by attaching these methods and properties to the $scope object.
    • With the introduction of the “controller as” syntax in AngularJS 1.3, the need to explicitly use $scope has been reduced. This syntax allows you to declare a controller on the view (e.g., ng-controller=”StoryboardCtrl as storyboard”) and then refer to its properties and methods directly in the template (e.g., {{storyboard.someProperty}}). This makes it clearer which part of the view is interacting with which controller instance.
    • The Controller consumes data from services (Models), prepares it for display in the View, and transmits data back to services for processing.
    • The Controller should be lightweight and focused on the specific View it controls. It should ideally be oblivious to the world around it unless explicitly told otherwise, meaning it shouldn’t have direct knowledge of the View it controls or other Controllers.

    Key aspects of MVVM in AngularJS:

    • Two-Way Data Binding: AngularJS provides two-way data binding between the View and the ViewModel ($scope or the controller instance with “controller as” syntax). When a property in the ViewModel changes, it is instantly reflected in the View, and vice versa (in cases like HTML forms where a user can directly manipulate a property). This reduces the need for manual DOM manipulation to keep the data synchronized.
    • Commands: The View can issue commands to the ViewModel (Controller) through mechanisms like ngClick or ngSubmit, which call methods defined on the $scope or the controller instance.
    • Change Notification: While not explicitly detailed as a separate step between Model and ViewModel in the AngularJS context of these sources, the two-way data binding and the use of services ensure that changes in the underlying data (managed by services/models) are eventually reflected in the View via the Controller. The digest cycle in AngularJS is the mechanism that detects changes in the model and updates the view accordingly.
    • Testability: The separation of concerns inherent in the MVVM pattern makes AngularJS applications more testable. You can test the ViewModel (Controller) independently of the View, and you can mock services (Models) to test the Controller’s logic.

    In summary, AngularJS leverages the MVVM pattern (with Controllers acting as ViewModels) to create structured and maintainable web applications. Data binding and the clear separation of Models (services), Views (HTML templates), and ViewModels (controllers) are central to this pattern, leading to more organized, testable, and efficient development. The “controller as” syntax further enhances the clarity and explicitness of the MVVM implementation in AngularJS.

    AngularJS Single-Page Application Routing and Navigation

    Routing and navigation are crucial aspects of building single-page applications (SPAs) with AngularJS, as they allow users to navigate between different views and states within the application without requiring a full page reload. AngularJS provides the ngRoute sub-module to implement client-side routing, enabling the application to respond to changes in the URL and render the appropriate content.

    Here’s a breakdown of routing and navigation in AngularJS based on the sources:

    • Purpose of Routing:
    • Routing allows you to define and navigate to unique states within your application based on the current URL.
    • It helps in deciding what to show and how to show it based on the application’s URL.
    • Routing enables features like deep-linking, allowing users to directly access specific parts of the application using a URL.
    • Core Components of ngRoute:
    • $routeProvider: This service is used to configure routes. You define URL patterns and associate them with specific views (templates) and controllers. Route configurations are typically done within the config block of your application module.
    • $route: This service listens to URL changes (specifically changes in $location.path) and coordinates with the ng-view directive to load and render the appropriate view and controller for the current route.
    • ng-view: This is a directive that acts as a placeholder in your main HTML template. When a route is matched by $route, ng-view is responsible for fetching the route’s template, compiling it with the route’s controller, and displaying the resulting view to the user. Only one ng-view directive can be declared per page with ngRoute.
    • $routeParams: This service is used to interpret and communicate URL parameters to the controller. When you define a route with parameters (e.g., /users/:userId), the values of these parameters are made available as properties on the $routeParams object within the associated controller.
    • Creating Routes with $routeProvider:
    • You configure routes using the when() method of $routeProvider. This method takes two arguments:
    • The path parameter (URL pattern): This defines the URL that the route will match against (e.g., /, /dashboard, /users/:userId). Named groups within the path, prefixed with a colon (e.g., :userId), define route parameters.
    • The route configuration object: This object defines how the matched route should be handled. Common properties include:
    • templateUrl: Specifies the path to the HTML template to be loaded for the route.
    • controller: Specifies the name of the controller function to be associated with the route.
    • controllerAs: Specifies an alias for the controller within the template when using the “controller as” syntax.
    • resolve: An object map that allows you to define dependencies that must be resolved before the route’s controller is instantiated. This is useful for preloading data. If a resolve property returns a promise, a $routeChangeSuccess event is fired upon resolution, and ngView instantiates the controller and renders the template. If the promise is rejected, a $routeChangeError event is fired.
    • You can also define a fallback route using the otherwise() method of $routeProvider, which specifies a route to redirect to if no other route matches the current URL.
    • Setting up Route Navigation:
    • The preferred way to navigate between routes is by using standard HTML anchor tags (<a>) with the href attribute set to the route’s URL, prefixed with a hash symbol (#) by default (e.g., <a href=”#/users”>Users</a>).
    • Using anchor tags is considered a best practice as it aligns with the browser’s expected behavior, such as allowing users to open links in a new tab.
    • While you can programmatically change routes using the $location service (and potentially ng-click), it is generally discouraged for navigation as it can break standard user experience patterns.
    • By default, routes work with a hashtag (#), but you can configure AngularJS to use HTML5 mode (without hashtags) or override the default delimiter, though this often requires server-side configuration.
    • Using Parameters with Routes:
    • You can define dynamic segments in your route paths using a colon followed by a parameter name (e.g., /users/:userId).
    • The values of these parameters are extracted from the URL and made available as properties on the $routeParams service within the associated controller. For example, if the URL is /users/123, $routeParams.userId will have the value ‘123’.
    • To create links with route parameters, you can use data binding within the href attribute of an anchor tag (e.g., <a href=”#/users/{{user.id}}”>View User</a>).
    • Route Events:
    • AngularJS broadcasts events on the $rootScope during the routing process, which you can listen to for performing actions like showing or hiding loading indicators.
    • Key route events include:
    • $routeChangeStart: Broadcasted before a route change begins.
    • $routeChangeSuccess: Broadcasted after a route is successfully changed and the controller is instantiated.
    • $routeChangeError: Broadcasted if there is an error during the route change process (e.g., a promise in resolve is rejected).
    • Testing Routes:
    • Testing routes involves injecting services like $location, $route, $templateCache, and $rootScope.
    • You manually navigate to a specific URL using $location.path(), trigger the digest cycle with $rootScope.$digest(), and then assert that the $route.current object has the expected controller, controllerAs, and templateUrl properties.
    • You might need to manually put templates into the $templateCache before navigating to a route in your tests.
    • Best Practices for Routing:
    • Your route structure should ideally mirror your application’s file structure. This makes it easier for developers to understand the organization of the application.
    • Favor using the resolve property to fetch resources via $routeParams whenever possible to keep controllers lean and focused on the view.
    • Use anchor tags (<a>) for navigation to maintain standard browser behavior.
    • Alternative Routing Solutions:
    • While ngRoute is simple to implement, it has limitations, such as only allowing one ng-view per page.
    • For more complex routing scenarios requiring multiple or nested views, the book recommends looking into ui-router, which is a powerful and full-featured routing solution.

    In summary, AngularJS provides a robust routing mechanism with ngRoute that allows you to build navigation into your single-page applications. By configuring routes with $routeProvider, using ng-view to render templates, and leveraging $routeParams for dynamic URLs, you can create well-structured and user-friendly web applications. Following best practices and considering alternative routers like ui-router for more advanced scenarios will further enhance your application’s architecture.

    By Amjad Izhar
    Contact: amjad.izhar@gmail.com
    https://amjadizhar.blog

  • Algorithms in a Nutshell Understand, Implement, Analyze Algorithms, Practical Aspects

    Algorithms in a Nutshell Understand, Implement, Analyze Algorithms, Practical Aspects

    This compilation of text, primarily from “Algorithms in a Nutshell, 2nd Edition,” provides a comprehensive guide to understanding, implementing, and analyzing various algorithms for a wide range of computational problems. The material covers fundamental concepts like sorting, searching, graph algorithms, and computational geometry, offering insights into their performance characteristics and practical considerations. Furthermore, it explores more advanced topics such as pathfinding in artificial intelligence, network flow algorithms, spatial data structures, and emerging algorithmic categories like approximation and probabilistic methods. The text also emphasizes practical aspects, including implementation details in multiple programming languages and methods for empirical performance evaluation. Ultimately, it serves as a reference for practitioners seeking to apply algorithmic solutions effectively.

    Algorithms Study Guide

    Study Questions

    Chapter 1: Thinking in Algorithms

    1. Describe the initial steps one should take when approaching a new algorithmic problem.
    2. Explain the concept of a “naïve solution” and its role in the algorithm design process.
    3. What are the key characteristics of an “intelligent approach” to solving algorithmic problems?

    Chapter 2: The Mathematics of Algorithms

    1. Define “size of a problem instance” and why it is important in algorithm analysis.
    2. Explain the significance of analyzing an algorithm’s performance in best, average, and worst-case scenarios. Provide an example of an algorithm where these cases differ.
    3. Describe the major “performance families” discussed in the chapter and provide an example of an algorithm that falls into each.
    4. What are “benchmark operations,” and how are they used in evaluating algorithm performance?
    5. Summarize the concept of sublinear time complexity and provide an example from the text.
    6. Discuss the challenges associated with comparing floating-point values for equality in algorithms.
    7. Explain the bisection method for finding the root of a function, referencing the example provided in the text.

    Chapter 3: Algorithm Building Blocks

    1. Outline the key components of the “Algorithm Template Format” and the “Pseudocode Template Format” as described in the chapter.
    2. What are some of the challenges and considerations when dealing with “Floating-Point Computation” in algorithms?
    3. Briefly explain the Greedy approach to algorithm design and provide an example from the text (like the partial convex hull).
    4. Describe the Divide and Conquer approach to algorithm design, using the convex hull computation as an example.
    5. Explain the steps involved in Graham’s Scan algorithm for convex hull computation.

    Chapter 5: Searching

    1. Compare and contrast sequential search and binary search in terms of their efficiency and the requirements for the data being searched.
    2. Explain the fundamental principles behind hash-based search and discuss the role of a hash function.
    3. Describe how a Bloom filter works and what are its key characteristics, including the possibility of false positives.
    4. Explain the structure and basic properties of a binary search tree.

    Chapter 6: Graph Algorithms

    1. Define the key components of a graph (vertices and edges) and differentiate between directed and undirected graphs.
    2. Describe the adjacency list and adjacency matrix representations of a graph, and discuss when each representation might be preferred.
    3. Explain the process of Depth-First Search (DFS) on a graph and its applications.
    4. Explain the process of Breadth-First Search (BFS) on a graph and how it differs from DFS.
    5. Describe the Single-Source Shortest Path problem and explain the core idea behind Dijkstra’s algorithm. What is a key limitation of Dijkstra’s algorithm?
    6. Explain how Dijkstra’s algorithm is adapted for dense graphs.
    7. Describe the Bellman-Ford algorithm and how it handles the possibility of negative edge weights. How can it detect negative cycles?
    8. Explain the All-Pairs Shortest Path problem and how the Floyd-Warshall algorithm solves it using dynamic programming.
    9. Describe the Minimum Spanning Tree (MST) problem and explain the Greedy approach used by Prim’s algorithm.

    Chapter 7: Path Finding in AI

    1. Explain the concept of game trees and their use in AI pathfinding.
    2. Describe the Minimax algorithm and its goal in game playing.
    3. Explain the NegMax algorithm and how it relates to the Minimax algorithm.
    4. Describe the AlphaBeta pruning technique and how it optimizes the Minimax/NegMax algorithms.
    5. Compare and contrast Depth-First Search and Breadth-First Search in the context of search trees in AI.
    6. Explain the A* search algorithm and the role of the heuristic function in guiding the search.

    Chapter 8: Network Flow Algorithms

    1. Define the key components of a flow network, including source, sink, edges, capacities, and flow.
    2. Explain the three key properties that a valid flow in a network must satisfy: capacity constraint, flow conservation, and skew symmetry.
    3. Describe the concept of an augmenting path in a flow network and how it is used to find the maximum flow.
    4. Briefly explain the application of network flow to the Bipartite Matching problem.
    5. What is the Minimum Cost Flow problem, and how does it extend the Maximum Flow problem?

    Chapter 9: Computational Geometry

    1. Explain the Convex Hull problem and describe a brute-force approach to finding a convex hull.
    2. Describe the Convex Hull Scan algorithm and its steps for finding the upper and lower convex hulls.
    3. Explain the LineSweep technique and how it can be used to find line segment intersections.
    4. Describe the Voronoi diagram of a set of points and its properties.

    Chapter 10: Spatial Tree Structures

    1. Explain the Nearest Neighbor Query problem and why a naïve linear scan might be inefficient for large datasets.
    2. Describe the structure of a k-d tree and how it partitions a multi-dimensional space.
    3. Explain how a k-d tree can be used to efficiently answer Nearest Neighbor Queries. What are some potential worst-case scenarios for the performance of k-d tree nearest neighbor search?
    4. Describe the Range Query problem and how k-d trees can be used to solve it.
    5. Explain the structure and purpose of a Quadtree.
    6. Explain the structure and purpose of an R-Tree, highlighting how it differs from a k-d tree in handling spatial data.

    Chapter 11: Emerging Algorithm Categories

    1. What is an Approximation Algorithm, and why might it be used instead of an exact algorithm?
    2. Briefly describe the Knapsack 0/1 problem and the Knapsack Unbounded problem.
    3. Explain the concept of Parallel Algorithms and the potential benefits of using multiple threads.
    4. What are Probabilistic Algorithms, and how do they differ from deterministic algorithms?

    Appendix A: Benchmarking

    1. Explain the purpose of benchmarking algorithms.
    2. Describe some common techniques for benchmarking algorithm performance, including the use of timers and multiple trials.
    3. Discuss the importance of considering factors like input size and configuration when benchmarking.

    Quiz

    1. What is the primary goal of algorithm analysis, and what mathematical concepts are often used in this process?
    2. Explain the difference between an algorithm with O(log n) time complexity and one with O(n^2) time complexity in terms of their scalability with increasing input size.
    3. In the context of hash-based search, what is a collision, and what are some common strategies for resolving collisions?
    4. Describe one practical application of Depth-First Search and one practical application of Breadth-First Search on graphs.
    5. What is the key distinguishing feature of Dijkstra’s algorithm that makes it suitable for finding shortest paths in certain types of graphs but not others?
    6. Explain the core principle behind dynamic programming as it is applied in the Floyd-Warshall algorithm for the All-Pairs Shortest Path problem.
    7. In the context of the A* search algorithm, what is the role of the heuristic function, and what properties should a good heuristic have?
    8. Describe the flow conservation property in a network flow algorithm and explain its significance.
    9. What is the fundamental idea behind the LineSweep technique in computational geometry, and for what type of problems is it typically used?
    10. Briefly explain how a k-d tree recursively partitions space and how this partitioning helps in nearest neighbor searches.

    Quiz Answer Key

    1. The primary goal of algorithm analysis is to predict the resources (like time and memory) required by an algorithm as a function of the input size. Mathematical concepts such as asymptotic notation (Big O, Omega, Theta) are commonly used to express the growth rate of these resources.
    2. An O(log n) algorithm has a time complexity that grows very slowly with increasing input size (n), typically halving the search space at each step. Conversely, an O(n^2) algorithm’s runtime grows quadratically with the input size, making it significantly slower for large inputs.
    3. In hash-based search, a collision occurs when two different keys produce the same hash value, mapping them to the same location in the hash table. Common collision resolution strategies include separate chaining (using linked lists) and open addressing (probing for an empty slot).
    4. Depth-First Search can be used for tasks like detecting cycles in a graph or topological sorting. Breadth-First Search is often used for finding the shortest path in an unweighted graph or for level-order traversal of a tree.
    5. The key distinguishing feature of Dijkstra’s algorithm is its greedy approach based on always selecting the unvisited vertex with the smallest known distance from the source. This makes it efficient for graphs with non-negative edge weights but can lead to incorrect results if negative edge weights are present.
    6. The core principle behind dynamic programming in the Floyd-Warshall algorithm is to break down the All-Pairs Shortest Path problem into smaller overlapping subproblems. It iteratively considers each vertex as a potential intermediate vertex in a shortest path between all other pairs of vertices, storing and reusing previously computed shortest path lengths.
    7. In A* search, the heuristic function provides an estimate of the cost from the current state to the goal state. A good heuristic should be admissible (never overestimate the true cost) and consistent (the estimated cost to reach the goal from a node should be no more than the cost of moving to a neighbor plus the estimated cost from that neighbor to the goal) to guarantee finding an optimal solution efficiently.
    8. The flow conservation property states that for every vertex in a flow network (except the source and sink), the total amount of flow entering the vertex must equal the total amount of flow leaving it. This property ensures that flow is neither created nor destroyed within the network.
    9. The fundamental idea behind the LineSweep technique is to move a virtual line across the geometric plane, processing the geometric objects (like line segments) in the order they are encountered by the line. This reduces a 2D problem to a 1D problem at each step, making it efficient for finding intersections or constructing Voronoi diagrams.
    10. A k-d tree recursively partitions a k-dimensional space by selecting one dimension at a time and splitting the data points based on the median value along that dimension. This hierarchical partitioning allows for efficient pruning of the search space during nearest neighbor queries by focusing on the regions of the tree that are closest to the query point.

    Essay Format Questions

    1. Discuss the trade-offs between different graph representations (adjacency lists vs. adjacency matrices) in the context of various graph algorithms discussed in the text. Consider factors such as space complexity, the cost of checking for the existence of an edge, and the efficiency of iterating over neighbors.
    2. Compare and contrast the Greedy approach used in Prim’s algorithm for Minimum Spanning Trees and Dijkstra’s algorithm for Single-Source Shortest Paths. Highlight the similarities and differences in their core logic and the problem constraints they address.
    3. Analyze the role of search algorithms in artificial intelligence, focusing on the differences between uninformed search (like BFS and DFS) and informed search (like A*). Discuss the importance of heuristic functions in guiding informed search and the implications of heuristic design on the efficiency and optimality of the search process.
    4. Explain how the concept of “divide and conquer” is applied in the context of the Convex Hull problem. Discuss the steps involved in a divide and conquer algorithm for finding the convex hull and its advantages over a simpler, iterative approach.
    5. Evaluate the significance of understanding algorithm performance families (e.g., logarithmic, linear, quadratic, exponential) in practical software development. Discuss how the choice of algorithm based on its performance characteristics can impact the scalability and efficiency of applications dealing with large datasets.

    Glossary of Key Terms

    • Algorithm: A well-defined sequence of steps or instructions to solve a problem or perform a computation.
    • Time Complexity: A measure of the amount of time an algorithm takes to run as a function of the size of the input. Often expressed using Big O notation.
    • Space Complexity: A measure of the amount of memory space an algorithm requires as a function of the size of the input.
    • Asymptotic Notation: Mathematical notation (Big O, Omega, Theta) used to describe the limiting behavior of a function, often used to classify the efficiency of algorithms.
    • Best Case: The scenario under which an algorithm performs most efficiently in terms of time or resources.
    • Worst Case: The scenario under which an algorithm performs least efficiently in terms of time or resources.
    • Average Case: The expected performance of an algorithm over all possible inputs of a given size.
    • Linear Time (O(n)): An algorithm whose execution time grows directly proportional to the size of the input.
    • Logarithmic Time (O(log n)): An algorithm whose execution time grows logarithmically with the size of the input, often seen in algorithms that divide the problem space in half at each step.
    • Quadratic Time (O(n^2)): An algorithm whose execution time grows proportionally to the square of the size of the input.
    • Greedy Algorithm: An algorithmic paradigm that makes locally optimal choices at each step with the hope of finding a global optimum.
    • Divide and Conquer: An algorithmic paradigm that recursively breaks down a problem into smaller subproblems until they become simple enough to solve directly, and then combines their solutions to solve the original problem.
    • Sequential Search: A simple search algorithm that iterates through a list of items one by one until the target item is found or the end of the list is reached.
    • Binary Search: An efficient search algorithm that works on sorted data by repeatedly dividing the search interval in half.
    • Hash Function: A function that maps input data of arbitrary size to a fixed-size output (hash value), often used in hash tables for efficient data retrieval.
    • Collision (Hashing): Occurs when two different input keys produce the same hash value.
    • Bloom Filter: A probabilistic data structure that can test whether an element is a member of a set. It may return false positives but never false negatives.
    • Binary Search Tree (BST): A tree data structure where each node has at most two children (left and right), and the left subtree of a node contains only nodes with keys less than the node’s key, and the right subtree contains only nodes with keys greater than the node’s key.
    • Graph: A data structure consisting of a set of vertices (nodes) connected by edges.
    • Directed Graph: A graph where the edges have a direction, indicating a one-way relationship between vertices.
    • Undirected Graph: A graph where the edges do not have a direction, indicating a two-way relationship between vertices.
    • Adjacency List: A graph representation where each vertex has a list of its neighboring vertices.
    • Adjacency Matrix: A graph representation where a matrix is used to represent the presence or absence of an edge between each pair of vertices.
    • Depth-First Search (DFS): A graph traversal algorithm that explores as far as possible along each branch before backtracking.
    • Breadth-First Search (BFS): A graph traversal algorithm that explores all the neighbor vertices at the present depth prior to moving on to the vertices at the next depth level.
    • Single-Source Shortest Path: The problem of finding the shortest paths from a single source vertex to all other vertices in a graph.
    • Dijkstra’s Algorithm: A greedy algorithm for finding the shortest paths from a single source vertex to all other vertices in a graph with non-negative edge weights.
    • Bellman-Ford Algorithm: An algorithm for finding the shortest paths from a single source vertex to all other vertices in a weighted graph, even if the graph contains negative edge weights (though it cannot handle negative cycles).
    • Negative Cycle: A cycle in a graph where the sum of the weights of the edges in the cycle is negative.
    • All-Pairs Shortest Path: The problem of finding the shortest paths between every pair of vertices in a graph.
    • Floyd-Warshall Algorithm: A dynamic programming algorithm for finding the shortest paths between all pairs of vertices in a weighted graph.
    • Minimum Spanning Tree (MST): A subset of the edges of a connected, undirected graph that connects all the vertices together, without any cycles and with the minimum possible total edge weight.
    • Prim’s Algorithm: A greedy algorithm for finding a minimum spanning tree for a weighted undirected graph.
    • Game Tree: A tree where nodes represent game states and edges represent possible moves in a game.
    • Minimax: A decision-making algorithm used in game theory to minimize the possible loss for a worst-case scenario.
    • AlphaBeta Pruning: An optimization technique for the Minimax algorithm that reduces the number of nodes evaluated in the game tree.
    • A Search:* An informed search algorithm that uses a heuristic function to guide the search for the shortest path.
    • Flow Network: A directed graph where each edge has a capacity and an associated flow.
    • Maximum Flow: The problem of finding the maximum amount of flow that can be sent from a source vertex to a sink vertex in a flow network without exceeding the capacity of any edge.
    • Augmenting Path: A path from the source to the sink in a residual graph that has available capacity, which can be used to increase the total flow in the network.
    • Bipartite Matching: The problem of finding a maximum set of edges without common vertices in a bipartite graph.
    • Computational Geometry: A branch of computer science that deals with algorithms for geometric problems.
    • Convex Hull: The smallest convex set that contains a given set of points.
    • LineSweep: A computational geometry technique that solves problems by sweeping a line across the plane.
    • Voronoi Diagram: A partition of a plane into regions based on the distance to points in a specific subset of the plane.
    • Spatial Tree: A tree data structure designed for efficient querying on spatial data, such as points or regions in a multi-dimensional space.
    • k-d Tree: A space-partitioning data structure for organizing points in a k-dimensional space.
    • Nearest Neighbor Query: The problem of finding the point in a dataset that is closest to a given query point.
    • Range Query: The problem of finding all points in a dataset that lie within a specified query range.
    • Quadtree: A tree data structure in which each internal node has exactly four children. Used for partitioning a two-dimensional space.
    • R-Tree: A tree data structure used for indexing multi-dimensional information such as geographical coordinates, rectangles or polygons.
    • Approximation Algorithm: An algorithm that finds a solution that is close to the optimal solution, especially for problems that are computationally hard to solve exactly in a reasonable amount of time.
    • Parallel Algorithm: An algorithm that can execute multiple operations simultaneously using multiple computing resources.
    • Probabilistic Algorithm: An algorithm that uses randomness as part of its logic.
    • Benchmarking: The process of running computer programs or parts of them, in order to assess their relative performance.

    Algorithms in a Nutshell, 2nd Edition: Key Concepts

    # Briefing Document: “Algorithms in a Nutshell, 2nd Edition” Excerpts

    **Date:** October 26, 2023

    **Source:** Excerpts from “Algorithms in a Nutshell, 2nd Edition.pdf”

    This briefing document summarizes the main themes and important ideas presented in the provided excerpts from “Algorithms in a Nutshell, 2nd Edition.” The excerpts cover fundamental concepts in algorithm design and analysis, specific algorithm categories (searching, graph algorithms, AI pathfinding, network flow, computational geometry, spatial tree structures), and emerging algorithm categories, along with practical considerations like benchmarking.

    ## Main Themes

    * **Problem Solving through Algorithms:** The book emphasizes a structured approach to problem-solving by first understanding the problem, exploring naïve solutions, and then developing more intelligent and efficient algorithmic approaches. Chapter 1 sets this stage.

    * **Mathematical Foundations of Algorithm Analysis:** A significant portion (Chapter 2) is dedicated to the mathematical tools needed to analyze algorithms. This includes understanding the size of a problem instance, the rate of growth of functions (Big O notation and performance families like constant, logarithmic, linear, polynomial, and exponential), best, average, and worst-case analysis, and identifying benchmark operations. The book uses examples like the Bisection method for root finding and the time taken for addition operations with varying input sizes to illustrate these concepts. For example, Table 2-3 shows the “Time (in milliseconds) to execute 10,000 add/plus invocations on random digits of size n,” demonstrating how execution time scales with input size.

    * **Fundamental Algorithm Building Blocks:** Chapter 3 introduces essential components and techniques used in algorithm design, including algorithm and pseudocode templates, empirical evaluation, considerations for floating-point computation (highlighting potential inaccuracies, as shown in the collinearity test example where 32-bit and 64-bit floats yield different results), common algorithmic approaches (like greedy, divide and conquer, dynamic programming, and backtracking), and provides an example algorithm (Graham Scan for convex hull).

    * **Core Algorithm Categories and Their Applications:** The excerpts delve into several key algorithm categories:

    * **Searching (Chapter 5):** Covers sequential search, binary search, hash-based search (including the importance of a good `hashCode()` method as illustrated in the Java example), Bloom filters (emphasizing their potential for false positives: “It may yet be the case that all bits are set but value was never added: false positive.”), and binary search trees.

    * **Graph Algorithms (Chapter 6):** Explores graph representations (adjacency lists and matrices, noting when each is more appropriate), fundamental graph traversal algorithms (Depth-First Search and Breadth-First Search), single-source shortest path algorithms (Dijkstra’s algorithm for both general and dense graphs, Bellman-Ford for handling negative edge weights), all-pairs shortest path (Floyd-Warshall), and minimum spanning tree algorithms (Prim’s). The text highlights the greedy nature of Dijkstra’s (“Dijkstra’s Algorithm conceptually operates in a greedy fashion…”) and Prim’s algorithms.

    * **Path Finding in AI (Chapter 7):** Focuses on algorithms used in artificial intelligence for pathfinding, including game trees, core concepts, Minimax, NegMax, AlphaBeta pruning (noting NegMax vs. AlphaBeta: “versus AlphaBeta, 188”), and search trees (revisiting DFS, BFS, and introducing A* search).

    * **Network Flow Algorithms (Chapter 8):** Discusses the concepts of network flow, maximum flow (illustrated with the Ford-Fulkerson method), bipartite matching (showing how to model it as a maximum flow problem), minimum cost flow, transshipment, transportation, assignment, and their relation to linear programming. The core idea of augmenting paths is central to maximum flow algorithms.

    * **Computational Geometry (Chapter 9):** Introduces problems in computational geometry, such as classifying problems, convex hull (mentioning different approaches like greedy and divide and conquer), line-segment intersection (using the line-sweep technique), and Voronoi diagrams. The condition for a right turn using a determinant is provided: “If cp < 0, then the three points determine a right turn…”.

    * **Spatial Tree Structures (Chapter 10):** Covers data structures designed for efficient spatial queries, including nearest neighbor queries, range queries, intersection queries, k-d trees, quadtrees, and R-trees. The text contrasts the naïve O(n) approach for nearest neighbor with the potential O(log n) of k-d trees (“This property will enable Nearest Neighbor to exhibit O(log n) performance…”).

    * **Emerging Algorithm Categories (Chapter 11):** Introduces variations and newer categories of algorithms, including approximation algorithms (like Knapsack 0/1 and Unbounded), parallel algorithms (briefly touching upon multithreading for quicksort), and probabilistic algorithms (like randomized quicksort). The description of the Knapsack 0/1 algorithm highlights its use of dynamic programming: “m[i][j] records maximum value using first i items without exceeding weight j.”

    * **Practical Considerations: Benchmarking (Appendix A):** Emphasizes the importance of empirically evaluating algorithm performance through benchmarking. It provides examples of shell scripts and Python code using the `timeit` module to measure execution times. It also discusses statistical interpretation of benchmarking results, including confidence intervals.

    ## Most Important Ideas and Facts

    * **Algorithm Analysis is Crucial:** Understanding the time and space complexity of algorithms is essential for choosing the most efficient solution for a given problem, especially as the input size grows. Big O notation provides a way to characterize this growth rate.

    * **Different Data Structures Suit Different Tasks:** The choice of data structure (e.g., adjacency list vs. matrix for graphs, hash table vs. binary search tree for searching, k-d tree vs. R-tree for spatial data) significantly impacts algorithm performance.

    * **Trade-offs Exist in Algorithm Design:** There are often trade-offs between different aspects of algorithm design, such as time complexity vs. space complexity, or exactness vs. approximation. Bloom filters, for instance, offer fast membership testing with a possibility of false positives, trading accuracy for speed and space efficiency.

    * **Greedy, Divide and Conquer, and Dynamic Programming are Powerful Paradigms:** These are recurring themes throughout the book, representing fundamental strategies for designing efficient algorithms for various problems.

    * **Floating-Point Arithmetic Has Limitations:** Computations involving floating-point numbers can introduce errors, which must be considered when designing and implementing algorithms that rely on precise comparisons.

    * **Spatial Data Structures Enable Efficient Spatial Queries:** For applications dealing with geometric data, specialized data structures like k-d trees and R-trees offer significant performance improvements over naive linear scans for tasks like nearest neighbor and range queries.

    * **Emerging Algorithm Categories Address New Challenges:** As computing evolves, new categories of algorithms are developed to tackle challenges like handling massive datasets (parallel algorithms) or finding solutions to computationally hard problems (approximation and probabilistic algorithms).

    * **Empirical Evaluation Complements Theoretical Analysis:** While theoretical analysis provides insights into algorithm scalability, benchmarking provides real-world performance data on specific hardware and software environments.

    This briefing provides a high-level overview of the key concepts and algorithms covered in the provided excerpts. The depth and breadth of topics suggest that “Algorithms in a Nutshell” aims to be a comprehensive resource for both understanding the fundamentals and exploring more advanced algorithmic techniques.

    Understanding Algorithms: Core Concepts and Applications

    Frequently Asked Questions About Algorithms

    • What is the fundamental approach to thinking in algorithms? Thinking in algorithms generally involves three stages. First, one must thoroughly understand the problem, including its inputs, expected outputs, and any constraints. Second, a naive solution might be considered, often being straightforward but potentially inefficient. Finally, the focus shifts to developing intelligent approaches, which are more efficient and tailored to the problem’s characteristics, often by leveraging common algorithmic patterns and data structures.
    • How is the efficiency of an algorithm typically analyzed? Algorithm efficiency is analyzed using mathematical concepts, primarily focusing on the rate of growth of the algorithm’s runtime or space requirements as the size of the input (n) increases. This is often expressed using Big O notation (e.g., O(n), O(log n), O(n^2)). Analysis can be performed for the best-case, average-case, and worst-case scenarios, providing a comprehensive understanding of performance. Key concepts include identifying benchmark operations and understanding performance families like constant, logarithmic, linear, and polynomial.
    • What are some common building blocks used in algorithm design? Algorithms are often constructed using fundamental building blocks. These include defining the format for algorithm templates and pseudocode to clearly express the steps involved. Empirical evaluation is also crucial to validate performance. Furthermore, understanding the nuances of floating-point computation and common algorithmic approaches like recursion, iteration, and divide-and-conquer are essential.
    • What are some fundamental searching algorithms and how do they differ? The text outlines several searching algorithms. Sequential search examines elements one by one. Binary search is more efficient for sorted data, repeatedly dividing the search interval in half. Hash-based search uses hash functions to map keys to indices in a hash table for fast lookups. Bloom filters are probabilistic data structures that can efficiently test whether an element is possibly in a set. Binary search trees provide a hierarchical structure for efficient searching, insertion, and deletion. Each algorithm has different performance characteristics and suitability depending on the data and the specific search requirements.
    • How are graphs represented and what are some basic graph traversal algorithms? Graphs can be represented using adjacency lists (efficient for sparse graphs) or adjacency matrices (better for dense graphs). Two fundamental graph traversal algorithms are Depth-First Search (DFS) and Breadth-First Search (BFS). DFS explores as far as possible along each branch before backtracking, while BFS explores all the neighbors of the current vertex before moving to the next level of neighbors. Both can be used for various graph-related tasks, such as finding paths and connected components.
    • What are some key path-finding algorithms in AI and graph theory, and what are their trade-offs? Path-finding algorithms aim to find the shortest or optimal path between nodes. Dijkstra’s algorithm finds the shortest paths from a single source vertex to all other vertices in a graph with non-negative edge weights. The Bellman-Ford algorithm can handle graphs with negative edge weights (but not negative cycles). Floyd-Warshall computes the shortest paths between all pairs of vertices. In AI, algorithms like Minimax, NegMax, and AlphaBeta are used for game tree search, while A* search is a heuristic search algorithm that efficiently finds the shortest path by balancing the cost to reach a node and an estimate of the cost to reach the goal. These algorithms have different time complexities and capabilities in handling various graph properties.
    • What is the concept of network flow and what are some related problems? Network flow deals with the movement of a commodity through a network of nodes connected by edges with capacities. A key problem is finding the maximum flow from a source to a sink while respecting edge capacities and flow conservation at intermediate vertices. Related problems include Bipartite Matching (finding the largest set of non-overlapping pairs between two sets of vertices), Minimum Cost Flow (finding a flow of a certain value with the minimum total cost), and Multi-Commodity Flow (where multiple commodities need to be routed through the same network).
    • How are spatial data structures like k-d trees and R-trees used for efficient spatial queries? Spatial data structures are designed to efficiently query geometric data. K-d trees partition a k-dimensional space by recursively dividing it along coordinate axes, enabling efficient nearest neighbor and range queries. R-trees are tree structures used for indexing multi-dimensional information such as rectangles and other polygons, supporting efficient intersection, containment, and nearest neighbor searches. These structures improve upon naive linear search by organizing data in a way that allows for pruning large portions of the search space.

    Algorithms in a Nutshell: Key Design Principles

    The book “Algorithms in a Nutshell” outlines several key principles behind the design and selection of algorithms. These principles are highlighted in the epilogue, which summarizes the concepts discussed throughout the book.

    Here are some of the fundamental algorithm principles discussed:

    • Know Your Data: Understanding the properties of your input data is crucial for selecting the most appropriate algorithm. The presence of specific characteristics, such as whether data is already sorted, uniformly distributed, or contains duplicates, can significantly impact an algorithm’s performance. For instance, Insertion Sort performs well on mostly sorted data, while Bucket Sort is efficient for uniformly distributed data. The book also notes that the absence of certain special cases in the data can simplify algorithm implementation.
    • Decompose a Problem into Smaller Problems: Many efficient algorithms rely on breaking down a problem into smaller, more manageable subproblems. Divide and Conquer strategies, exemplified by Quicksort and Merge Sort, follow this principle by recursively dividing the problem until a base case is reached. The solutions to the subproblems are then combined to solve the original problem. Dynamic Programming is presented as a variation where subproblems are solved only once and their results stored for future use.
    • Choose the Right Data Structure: The selection of appropriate data structures is critical for achieving optimal algorithm performance. As the book states, with the right data structure, many problems can be solved efficiently. For example, using a binary heap for a priority queue allows for O(log n) removal of the minimum priority element. The choice between an adjacency list and an adjacency matrix for graph representation depends on the graph’s sparsity and significantly affects the performance of graph algorithms.
    • Make the Space versus Time Trade-Off: Algorithms can often be optimized by using extra storage to save computation time. Prim’s Algorithm utilizes additional arrays to efficiently track visited vertices and their distances, improving its performance. Bucket Sort, despite its high memory requirements, can achieve linear time complexity for uniformly distributed data by using extra storage.
    • Construct a Search: For problems where no direct solution is apparent, formulating the problem as a search over a large graph can be a viable approach, particularly in artificial intelligence. Algorithms like Depth-First Search, Breadth-First Search, and A* Search explore the solution space to find a desired outcome. However, the book cautions against using search algorithms with exponential behavior when more efficient computational alternatives exist.
    • Reduce Your Problem to Another Problem: Problem reduction involves transforming a given problem into a different problem for which an efficient solution is already known. The book gives the example of finding the fourth largest element by first sorting the list. In computational geometry, the convex hull can be derived from the Voronoi diagram. Furthermore, various network flow problems can be reduced to linear programming, although specialized network flow algorithms often offer better performance.
    • Writing Algorithms Is Hard—Testing Algorithms Is Harder: The process of developing and verifying algorithms, especially non-deterministic ones or those involving search, can be challenging. Testing often involves ensuring reasonable behavior rather than a specific outcome, particularly for algorithms in AI.
    • Accept Approximate Solutions When Possible: In some scenarios, especially when dealing with complex problems, accepting a solution that is close to the optimal one can lead to more efficient algorithms . Approximation algorithms, discussed in Chapter 11, aim to find near-optimal solutions in less time than it would take to find an exact solution .
    • Add Parallelism to Increase Performance: Utilizing parallel computing by creating multiple computational processes can significantly improve the performance of algorithms . The book illustrates this with a multithreaded implementation of Quicksort. However, it also notes that there is overhead associated with using threads, and parallelism should be applied judiciously.

    These principles provide a framework for understanding how algorithms are designed and how to approach problem-solving in an efficient and effective manner. By considering these concepts, one can better select, implement, and even develop algorithms tailored to specific needs and data characteristics.

    Sorting Algorithms: Concepts, Techniques, and Analysis

    The book “Algorithms in a Nutshell” dedicates Chapter 4 to Sorting Algorithms, emphasizing their fundamental role in simplifying numerous computations and tasks. The early research in algorithms heavily focused on efficient sorting techniques, especially for large datasets. Even with today’s powerful computers, sorting large numbers of items remains a common and important task.

    When discussing sorting algorithms, certain terminology is used. A collection of comparable elements A is presented to be sorted in place. A[i] or a_i refers to the ith element, with the first element being A. A[low, low + n) denotes a sub-collection of n elements, while A[low, low + n] contains n + 1 elements. The goal of sorting is to reorganize the elements such that if A[i] < A[j], then i < j. Duplicate elements must be contiguous in the sorted collection. The sorted collection must also be a permutation of the original elements.

    The collection to be sorted might be in random access memory (RAM) as pointer-based or value-based storage. Pointer-based storage uses an array of pointers to the actual data, allowing for sorting of complex records efficiently. Value-based storage packs elements into fixed-size record blocks, better suited for secondary or tertiary storage. Sorting algorithms update the information in both storage types so that A[0, n) is ordered.

    For a collection to be sorted, its elements must admit a total ordering. For any two elements p and q, exactly one of p = q, p < q, or p > q must be true. Commonly sorted types include integers, floating-point values, and characters. Composite elements like strings are sorted lexicographically. The algorithms typically assume a comparator function, cmp(p, q), which returns 0 if p = q, a negative number if p < q, and a positive number if p > q.

    Stable sorting is a property where if two elements a_i and a_j are equal according to the comparator in the original unordered collection and i < j, their relative order is maintained in the sorted set. Merge Sort is an example of a sorting algorithm that guarantees stability.

    The choice of sorting algorithm depends on several qualitative criteria:

    • For only a few items or mostly sorted items, Insertion Sort is suitable.
    • If concerned about worst-case scenarios, Heap Sort is a good choice.
    • For good average-case behavior, Quicksort is often preferred.
    • When items are drawn from a uniform dense universe, Bucket Sort can be efficient.
    • If minimizing code is a priority, Insertion Sort is simple to implement.
    • When a stable sort is required, Merge Sort should be used.

    Chapter 4 details several sorting algorithms:

    • Transposition Sorting: This family includes algorithms like Selection Sort and Bubble Sort, but the book focuses on Insertion Sort.
    • Insertion Sort repeatedly inserts an element into its correct position within a growing sorted region. It has a best-case performance of O(n) when the input is already sorted and an average and worst-case performance of O(n²). It is efficient for small or nearly sorted collections. The book provides C implementations for both pointer-based and value-based storage. Empirical results show the quadratic behavior, even with optimizations for value-based data. As noted in our previous discussion on algorithm principles, Insertion Sort’s efficiency on nearly sorted data aligns with the principle of “Know Your Data.”
    • Selection Sort repeatedly selects the largest value from an unsorted range and swaps it with the rightmost element of that range. It has a worst-case, average-case, and best-case performance of O(n²) and is considered the slowest of the described algorithms. It serves as a basis for understanding the principle behind Heap Sort.
    • Heap Sort: This algorithm uses a binary heap data structure to sort elements.
    • Heap Sort has a best, average, and worst-case performance of O(n log n). It involves building a heap from the input and then repeatedly extracting the maximum element and placing it in its sorted position. The heapify operation is central to this algorithm. The book provides a C implementation and compares recursive and non-recursive versions. Heap Sort is recommended when concerned about worst-case scenarios.
    • Partition-Based Sorting: The primary example is Quicksort.
    • Quicksort is a Divide and Conquer algorithm that selects a pivot element to partition the array into two subarrays, recursively sorting each. It has an average and best-case performance of O(n log n), but its worst-case performance is O(n²). The choice of pivot significantly impacts its performance. The book provides a C implementation. Various optimizations and enhancements to Quicksort exist, making it a popular choice in practice. The concept of decomposing a problem into smaller problems, as highlighted in our earlier discussion of algorithm principles, is central to Quicksort. A multithreaded version of Quicksort is also mentioned in the context of parallel algorithms, demonstrating how parallelism can be added to increase performance, another algorithm principle we discussed.
    • Sorting without Comparisons: Bucket Sort is presented as an algorithm that can achieve linear O(n) performance if certain conditions are met.
    • Bucket Sort works by partitioning the input into a set of ordered buckets using a hash function. Each bucket is then sorted (typically using Insertion Sort), and the elements are collected in order. It requires a uniform distribution of the input data and an ordered hash function. The book provides a C implementation using linked lists for buckets. Performance is highly dependent on the number of buckets and the distribution of data. As per our previous discussion, Bucket Sort exemplifies the space-versus-time trade-off and the principle of “Know Your Data”.
    • Sorting with Extra Storage: Merge Sort is the main algorithm discussed in this category.
    • Merge Sort is a Divide and Conquer algorithm that divides the collection into halves, recursively sorts them, and then merges the sorted halves. It has a best, average, and worst-case performance of O(n log n) and requires O(n) extra storage in its efficient implementation. It is well-suited for sorting external data and guarantees stability. The book includes a Java implementation for external Merge Sort using memory mapping. Merge Sort exemplifies the Divide and Conquer principle discussed earlier.

    Chapter 4 also includes string benchmark results comparing the performance of these sorting algorithms on random permutations of 26-letter strings and “killer median” data designed to make Quicksort perform poorly. These results highlight the practical implications of the theoretical performance analysis.

    Finally, the chapter discusses analysis techniques for sorting algorithms, emphasizing the importance of understanding best-case, worst-case, and average-case performance. It also touches on the theoretical lower bound of O(n log n) for comparison-based sorting algorithms, which is proven using the concept of binary decision trees. This theoretical understanding helps in appreciating why algorithms like Merge Sort and Heap Sort are considered efficient in the general case. The summary table in the epilogue (Table 12-1) reinforces the key characteristics and performance of each sorting algorithm discussed in this chapter.

    Algorithms in a Nutshell: Searching Algorithms

    Chapter 5 of “Algorithms in a Nutshell” focuses on Searching Algorithms, addressing two fundamental queries on a collection C of elements:

    • Existence: Determining if C contains a target element t.
    • Associative lookup: Retrieving information associated with a target key value k in C.

    The choice of search algorithm is heavily influenced by how the data is structured and the nature of the search operations. For instance, sorting a collection beforehand (as discussed in Chapter 4 and our previous conversation) can significantly improve search performance, although maintaining a sorted collection has its own costs, especially with frequent insertions and deletions. Ultimately, the performance of a search algorithm is judged by the number of elements it inspects while processing a query.

    The book provides the following guide for selecting the best search algorithm based on different scenarios:

    • For small collections or when the collection is only accessible sequentially (e.g., via an iterator), Sequential Search is the simplest and often the only applicable method.
    • When the collection is an unchanging array and you want to conserve memory, Binary Search is recommended.
    • If the elements in the collection change frequently (dynamic membership), consider Hash-Based Search and Binary Search Tree due to their ability to handle modifications to their data structures.
    • When you need dynamic membership and the ability to process elements in sorted order, a Binary Search Tree is the appropriate choice.

    It’s also crucial to consider any upfront preprocessing required by the algorithm to structure the data before handling search queries. The goal is to choose a structure that not only speeds up individual queries but also minimizes the overall cost of maintaining the collection in the face of dynamic access and multiple queries.

    The algorithms discussed in Chapter 5 assume a universe U of possible values, from which the elements in the collection C and the target element t are drawn. The collection C can contain duplicate values. When C allows indexing of arbitrary elements, it is referred to as an array A, with A[i] representing the ith element. The value null is used to represent an element not in U, and searching for null is generally not possible.

    Here are the searching algorithms detailed in Chapter 5:

    • Sequential Search:
    • Also known as linear search, this is the simplest approach, involving a brute-force examination of each element in the collection C until the target value t is found or all elements have been checked. The order of access doesn’t matter; it can be applied to both indexed collections (arrays) and collections accessible via a read-only iterator.
    • Input/Output: A nonempty collection C of n > 0 elements and a target value t. Returns true if C contains t, and false otherwise.
    • Context: Useful when no prior information about the collection’s order is available or when the collection can only be accessed sequentially through an iterator. It places the fewest restrictions on the type of elements being searched.
    • Summary: Best: O(1) (when the target is the first element), Average, Worst: O(n) (when the target is not present or is the last element).
    • Principles: Brute Force.
    • The chapter provides pseudocode and code examples in Python and Java for both indexed and iterable collections. Empirical performance data (Table 5-1) illustrates the linear relationship between collection size and search time. As noted in our previous discussion, for small collections, Sequential Search offers the simplest implementation, aligning with the principle of choosing the most appropriate algorithm based on the data.
    • Binary Search:
    • This algorithm offers better performance than Sequential Search by requiring the collection A to be already sorted. It works by repeatedly dividing the sorted collection in half. If the middle element matches the target t, the search is complete. Otherwise, the search continues in the left or right half depending on whether t is less than or greater than the middle element.
    • Input/Output: An indexed and totally ordered collection A. Returns true if t exists in A, and false otherwise.
    • Context: Efficient for searching through ordered collections, requiring a logarithmic number of probes in the worst case. It’s best suited for static, unchanging collections stored in arrays for easy navigation.
    • Summary: Best: O(1) (when the target is the middle element), Average, Worst: O(log n).
    • Principles: Divide and Conquer.
    • Pseudocode and a Java implementation using the java.util.Comparable<T> interface are provided. Binary Search exemplifies the principle of “Decompose a Problem into Smaller Problems” through its halving strategy. It also aligns with the recommendation to use it when the collection is an array that doesn’t change and memory conservation is desired.
    • Hash-Based Search:
    • This approach uses a hash function to transform characteristics of the searched-for item into an index within a hash table H. It generally offers better average-case performance than Sequential and Binary Search for larger, potentially unordered collections.
    • Input/Output: A computed hash table H and a target element t. Returns true if t exists in the linked list stored by H[h] (where h = hash(t)), and false otherwise. The original collection C does not need to be ordered.
    • Context: Suitable for large collections that are not necessarily ordered. The performance depends on the design of the hash function and the strategy for handling collisions (when multiple elements have the same hash value). A common collision resolution technique is using linked lists at each hash index.
    • Summary: Best, Average: O(1) (assuming a good hash function and few collisions), Worst: O(n) (in the case of many collisions where all elements hash to the same bin, leading to a linear search through a linked list).
    • Principles: Hash.
    • The chapter discusses the general pattern of Hash-Based Search, concerns like hash function design and collision handling, and provides pseudocode for loading a hash table and searching. An example using the hashCode() method of Java’s String class and a modulo operation to fit within the hash table size is given. The concept of a perfect hash function (guaranteeing no collisions for a specific set of keys) is also briefly mentioned as a variation. Different collision handling techniques, such as open addressing (linear probing, quadratic probing, double hashing), are discussed as variations that avoid linked lists but can lead to clustering. Hash-Based Search demonstrates the principle of “Choose the Right Data Structure,” where a well-designed hash table can provide efficient average-case search performance.
    • Bloom Filter:
    • This is a probabilistic data structure that can tell you if an element might be in a set. Unlike other search algorithms, it has a chance of giving a false positive (reporting that an element is present when it is not), but it will never give a false negative (it will always correctly identify an element that is not present).
    • Input/Output: A Bloom Filter data structure and a target element t. Returns true if t might be in the set, and false if t is definitely not in the set.
    • Context: Useful when it’s acceptable to have a small probability of false positives in exchange for significantly reduced storage space compared to storing the full set of values.
    • Summary: Insertion and search take O(k) time, where k is the number of hash functions used, which is considered constant. The storage required is fixed and won’t increase with the number of stored values.
    • Principles: False Positive.
    • The chapter explains the working mechanism of a Bloom Filter, which involves using multiple hash functions to set bits in a bit array. It highlights the trade-off between the size of the bit array, the number of hash functions, and the false positive rate. The Bloom Filter exemplifies the principle of accepting approximate solutions when possible.
    • Binary Search Tree:
    • This is a tree-based data structure where each node has a value greater than all nodes in its left subtree and less than all nodes in its right subtree. This structure allows for efficient searching, insertion, and deletion of elements while maintaining sorted order.
    • Input/Output: A Binary Search Tree containing elements from a collection C where each element has a comparable key. Search operations typically return true/false or the node with the matching key.
    • Context: Suitable for dynamic collections where elements are frequently inserted or deleted and where elements need to be accessed in sorted order.
    • Summary: Best: O(1) (when the target is the root), Average: O(log n) (for balanced trees), Worst: O(n) (for skewed trees where the tree resembles a linked list). AVL Binary Search Tree, a self-balancing variant, guarantees O(log n) performance for all cases.
    • Principles: Binary Tree. Balanced (for AVL).
    • The chapter discusses the basic properties of a Binary Search Tree and a specific implementation of a self-balancing AVL tree as a “Solution”. AVL trees maintain balance through rotations, ensuring logarithmic performance for insertions, deletions, and searches. Binary Search Trees and their balanced variants like AVL trees demonstrate the principle of choosing the right data structure to achieve efficient performance for dynamic operations and sorted access.

    In summary, Chapter 5 provides a comprehensive overview of fundamental searching algorithms, highlighting their principles, performance characteristics, and suitability for different scenarios, further emphasizing the importance of understanding your data and choosing the right data structure and algorithm for the task, as discussed in the epilogue of the book.

    Algorithms in a Nutshell: Graph Algorithms

    Chapter 6 of “Algorithms in a Nutshell” delves into Graph Algorithms, highlighting graphs as fundamental structures for representing complex structured information. The chapter investigates common ways to represent graphs and associated algorithms that frequently arise.

    Fundamental Concepts:

    • A graph G = (V, E) is defined by a set of vertices V and a set of edges E over pairs of these vertices. The terms “node” and “link” might be used elsewhere to represent the same information.
    • The book focuses on simple graphs that avoid self-edges and multiple edges between the same pair of vertices.
    • Three common types of graphs are discussed:
    • Undirected, unweighted graphs: Model symmetric relationships between vertices.
    • Directed graphs: Model relationships where the direction matters.
    • Weighted graphs: Model relationships with an associated numeric weight. Weights can represent various information like distance, time, or cost. The most structured type is a directed, weighted graph.
    • Problems involving graphs often relate to finding paths between vertices. A path is described as a sequence of vertices, and in directed graphs, the path must respect the direction of the edges. A cycle is a path that includes the same vertex multiple times. A graph is connected if a path exists between any two pairs of vertices.

    Graph Representation:

    • Two common ways to store graphs are discussed:
    • Adjacency Lists: Each vertex maintains a linked list of its adjacent vertices, often storing the weight of the edge as well. This representation is suitable for sparse graphs where the number of edges is much smaller than the potential number of edges. When using an adjacency list for an undirected graph, each edge (u, v) appears twice, once in u’s list and once in v’s list.
    • Adjacency Matrix: An n-by-n matrix A (where n is the number of vertices) where A[i][j] stores the weight of the edge from vertex i to vertex j. If no edge exists, a special value (e.g., 0, -1, or -∞) is used. Checking for the existence of an edge is constant time with an adjacency matrix, but finding all incident edges to a vertex takes more time in sparse graphs compared to adjacency lists. Adjacency matrices are more suitable for dense graphs where nearly every possible edge exists. For undirected graphs, the adjacency matrix is symmetric (A[i][j] = A[j][i]).
    • The book’s implementation uses a C++ Graph class with an adjacency list representation using the C++ Standard Template Library (STL).

    Graph Operations:

    • The chapter outlines several categories of operations on graphs:
    • Create: Constructing a graph with a given set of vertices, either directed or undirected.
    • Inspect: Determining if a graph is directed, finding incident edges, checking for the existence of a specific edge, and retrieving edge weights. Iterators can be used to access neighboring edges.
    • Update: Adding or removing edges from a graph. Adding or removing vertices is also possible but not needed by the algorithms discussed in this chapter.

    Graph Exploration Algorithms:

    • Two fundamental search strategies for exploring a graph are discussed:
    • Depth-First Search (DFS): Explores as far as possible along each branch before backtracking. It uses a recursive dfsVisit(u) operation and colors vertices white (not visited), gray (visited, may have unvisited neighbors), and black (visited with all neighbors visited). DFS computes a pred[v] array to record the predecessor vertex, allowing for path recovery from the source.
    • Input/Output: A graph G = (V, E) and a source vertex s ∈ V. Produces the pred[v] array.
    • Context: Requires O(n) overhead for storing vertex colors and predecessor information.
    • Summary: Best, Average, Worst: O(V + E).
    • Variations: For unconnected graphs, multiple dfsVisit calls can process all vertices, resulting in a depth-first forest.
    • Breadth-First Search (BFS): Systematically visits all vertices at a given distance (in terms of number of edges) from the source before moving to vertices at the next distance level. It uses a queue to maintain the vertices to be processed. BFS computes dist[v] (shortest path distance in edges) and pred[v].
    • Input/Output: A graph G = (V, E) and a source vertex s ∈ V. Produces dist[v] and pred[v] arrays.
    • Context: Requires O(V) storage for the queue. Guaranteed to find the shortest path in terms of edge count.
    • Summary: Best, Average, Worst: O(V + E).

    Shortest Path Algorithms:

    • The chapter covers algorithms for finding shortest paths in weighted graphs:
    • Single-Source Shortest Path: Given a source vertex s, compute the shortest path to all other vertices.
    • Dijkstra’s Algorithm: Finds the shortest paths from a single source to all other vertices in a directed, weighted graph with non-negative edge weights. It uses a priority queue (PQ) to maintain vertices by their current shortest distance from the source.
    • Input/Output: A directed, weighted graph G = (V, E) with non-negative edge weights and a source vertex s ∈ V. Produces dist[] (shortest distances) and pred[] (predecessor vertices).
    • Summary: Best, Average, Worst: O((V + E) * log V) (when using a binary heap for the priority queue).
    • Dijkstra’s Algorithm for Dense Graphs: An optimized version for dense graphs represented by an adjacency matrix, which avoids using a priority queue. It iteratively finds the unvisited vertex with the smallest distance.
    • Summary: Best, Average, Worst: O(V² + E).
    • Bellman–Ford Algorithm: Can handle directed, weighted graphs with negative edge weights, as long as there are no negative cycles (cycles whose edge weights sum to a negative value).
    • Summary: Best, Average, Worst: O(V * E).
    • Comparison of Single-Source Shortest-Path Options: The chapter provides benchmark results (Tables 6-1, 6-2, 6-3) comparing the performance of Dijkstra’s (with priority queue and optimized for dense graphs) and Bellman-Ford on different types of graphs (benchmark, dense, and sparse). It highlights that Dijkstra’s with a priority queue generally performs best on sparse graphs, the optimized Dijkstra’s does well on dense graphs, and Bellman-Ford is suitable when negative edge weights are present but performs poorly on dense graphs compared to Dijkstra’s.
    • All-Pairs Shortest Path: Compute the shortest path between all pairs of vertices in the graph.
    • Floyd–Warshall Algorithm: Uses Dynamic Programming to compute the shortest distances between all pairs of vertices in a directed, weighted graph with positive edge weights. It computes an n-by-n distance matrix dist and a predecessor matrix pred.
    • Input/Output: A directed, weighted graph G = (V, E) with positive edge weights. Produces dist[][] and pred[][] matrices.
    • Summary: Best, Average, Worst: O(V³).

    Minimum Spanning Tree (MST) Algorithms:

    • Given an undirected, connected, weighted graph, find a subset of edges that connects all vertices with the minimum total weight.
    • Prim’s Algorithm: A Greedy algorithm that builds the MST one edge at a time by iteratively adding the lowest-weight edge connecting a vertex in the MST to a vertex outside of it. It uses a priority queue to store vertices outside the MST, prioritized by the weight of the lightest edge connecting them to the MST.
    • Input/Output: An undirected graph G = (V, E). Produces an MST encoded in the pred[] array.
    • Summary: Best, Average, Worst: O((V + E) * log V).
    • Kruskal’s Algorithm: Another greedy algorithm that builds the MST by processing all edges in order of weight (from smallest to largest) and adding an edge if it doesn’t create a cycle. It uses a “disjoint-set” data structure.
    • Summary: O(E log V).

    Final Thoughts on Graphs:

    • The choice between using an adjacency list or an adjacency matrix largely depends on whether the graph is sparse or dense. Adjacency matrices require O(n²) storage, which can be prohibitive for large sparse graphs. Traversing large matrices in sparse graphs can also be inefficient.
    • The performance of some graph algorithms varies based on the graph’s density. For sparse graphs (|E| is O(V)), algorithms with a time complexity of O((V + E) log V) are generally more efficient, while for dense graphs (|E| is O(V²)), algorithms with O(V² + E) might be better. The break-even point is when |E| is on the order of O(V²/log V).

    Chapter 6 provides a solid foundation in graph algorithms, covering essential algorithms for searching, finding shortest paths, and determining minimum spanning trees, along with considerations for graph representation and performance based on graph density. This knowledge aligns with the principles discussed in the epilogue, such as choosing the right data structure (adjacency list vs. matrix) and understanding the impact of input data characteristics.

    Spatial Tree Structures: k-d Trees, Quadtrees, R-Trees

    Chapter 10 of “Algorithms in a Nutshell” focuses on Spatial Tree Structures, which are designed for efficiently modeling two-dimensional (and by extension, n-dimensional) data over the Cartesian plane to support powerful search queries beyond simple membership. These structures partition data in space to improve the performance of operations like search, insert, and delete. The chapter emphasizes three main types of spatial tree structures: k-d Trees, Quadtrees, and R-Trees.

    Types of Spatial Tree Structures:

    • k-d Tree:
    • A recursive binary tree structure that subdivides a k-dimensional plane along the perpendicular axes of the coordinate system. The book primarily discusses two-dimensional k-d trees.
    • Each node in a 2-d tree contains a point and either an x or y coordinate label that determines the partitioning orientation.
    • The root node represents the entire plane, and each subsequent level partitions the region based on the coordinate label of the node.
    • k-d trees are used to efficiently support nearest neighbor queries and range queries. For nearest neighbor queries, the tree structure allows discarding subtrees that are demonstrably too far to contain the closest point, achieving an average performance of O(log n) for well-distributed points. Range queries, which ask for all points within a given rectangular region, can be performed in O(n¹⁻¹/ᵈ + r) on average, where d is the number of dimensions and r is the number of reported points.
    • A limitation of k-d trees is that they cannot be easily balanced, and deleting points is complex due to the structural information they represent. The efficiency can also degrade in higher dimensions; some believe they are less efficient than a straight comparison for more than 20 dimensions.
    • Quadtree:
    • Another tree structure used for partitioning a two-dimensional space. The book focuses on point-based quadtrees, where each node represents a square region and can store up to four points.
    • If a region becomes full, it is subdivided into four equal-sized quadrants, creating four child nodes. The shape of the tree depends on the order in which points are added.
    • Quadtrees are effective for range queries, where they can identify points within a query rectangle . If a quadtree region is wholly contained by the query, all points within that region and its descendants can be efficiently included in the result. They are also used for collision detection by finding intersections among objects in the plane.
    • The summary for quadtrees indicates a Best, Average, Worst case performance of O(log n). However, a degenerate case is shown where the structure can become linear if points are added in a specific order.
    • R-Tree:
    • A height-balanced tree structure where each node can contain up to M links to child nodes, and leaf nodes store up to M n-dimensional spatial objects (in the book’s examples, these are typically rectangles in two dimensions).
    • Interior nodes store the bounding boxes that encompass all the rectangles in their descendant nodes. The root node’s bounding box covers all rectangles in the tree.
    • R-trees are designed to efficiently support nearest neighbor queries, range queries (locating objects that overlap with a target query rectangle), and intersection queries. They also support insertion and deletion operations.
    • A key advantage of R-trees is their ability to handle data that is too large to fit in main memory, making them suitable for secondary storage due to their page-friendly structure (similar to B-trees).
    • The summary for R-trees indicates a Best, Average case performance of O(log<0xE2><0x82><0x98> n) and a Worst case of O(n), where m is a parameter defining the minimum number of children per node.

    Applications and Concepts:

    • These spatial tree structures are fundamental for various applications including:
    • Nearest Neighbor Queries: Finding the closest point in a set to a given query point (e.g., finding the nearest gas station).
    • Range Queries: Retrieving all points or objects within a specified spatial region (e.g., selecting all restaurants within a map view).
    • Intersection Queries: Identifying intersections between spatial objects (e.g., collision detection in games or VLSI design rule checking).
    • The choice of which spatial tree structure to use depends on the specific application and the nature of the data (e.g., point data vs. region data), as well as the frequency of insertions and deletions.
    • The concept of partitioning space efficiently is central to these structures, allowing algorithms to avoid examining large portions of the data during a query, thus improving performance compared to brute-force approaches.

    Chapter 10 demonstrates how these tree-based structures extend the principles of binary search trees to handle spatial data, providing efficient solutions for common geometric queries.

    While not a tree structure, Chapter 9 also mentions the Voronoi diagram as a geometric structure that divides a plane into regions based on proximity to a set of points. Once computed, the Voronoi diagram can be used to solve problems like finding the convex hull. The construction of a Voronoi diagram itself can utilize a line-sweep technique, as discussed for other computational geometry problems.

    By Amjad Izhar
    Contact: amjad.izhar@gmail.com
    https://amjadizhar.blog

  • Comprehensive Java Full Course with Advanced Java Features for Java Developers having Great Career Prospects

    Comprehensive Java Full Course with Advanced Java Features for Java Developers having Great Career Prospects

    This comprehensive Java resource covers fundamental programming concepts through advanced topics. It begins with basic syntax, control flow (loops and conditionals), and object-oriented programming principles like encapsulation, inheritance, polymorphism, and abstraction. The material further explores advanced Java features such as exception handling, multithreading, collections (ArrayList), and string manipulation. File operations, interfaces, abstract classes, and generic types are also explained. The resource also discusses data structures (stacks, queues, linked lists, arrays, sorting, searching) and provides insights into becoming a Java developer, including necessary skills and career prospects. Finally, it includes examples of Java projects like an ATM, text editor, and signup form.

    Java Programming Study Guide

    Quiz

    1. Explain the concept of platform independency in Java. Why is Java known as a “Write Once, Run Anywhere” language?
    2. What is multi-threading in Java, and what are its benefits? Provide a brief example of a scenario where multi-threading would be advantageous.
    3. Describe the difference between pre-increment/decrement and post-increment/decrement unary operators in Java. Illustrate with a short code snippet.
    4. Explain the purpose of relational operators in Java. Give three examples of relational operators and describe the conditions under which they would evaluate to true.
    5. What is the primary function of logical operators in Java? Describe the behavior of the && (AND) and || (OR) operators.
    6. Explain the functionality of the ternary operator in Java. Provide a simple example demonstrating its usage and the benefit it offers.
    7. Describe the difference in behavior between a while loop and a do-while loop in Java. When might you choose to use one over the other?
    8. What is the purpose of the break statement within a switch block in Java? What happens if a break statement is omitted?
    9. Explain the concept of nested if statements in Java. Provide a simple scenario where using a nested if statement would be appropriate.
    10. What is the role of control statements in Java? List three different types of control statements discussed in the source material.

    Quiz Answer Key

    1. Platform independency in Java means that a Java program can run on any operating system that has a compatible Java Runtime Environment (JRE) installed. Java achieves this because, upon compilation, the code is translated into platform-independent bytecode, rather than machine-specific code. This bytecode can then be executed by any JRE, making Java a “Write Once, Run Anywhere” language.
    2. Multi-threading in Java is a feature that allows a program to execute multiple parts (threads) concurrently. The benefit of multi-threading is that it can improve performance by utilizing the same memory and resources to perform multiple tasks simultaneously. For example, while typing in a word processor, a separate thread can be used to perform spell checking in the background.
    3. Pre-increment/decrement operators modify the value of a variable before it is used in an expression, while post-increment/decrement operators modify the value after it has been used.
    4. int a = 5;
    5. int b = ++a; // pre-increment: a becomes 6, b is assigned 6
    6. int c = 5;
    7. int d = c++; // post-increment: d is assigned 5, then c becomes 6
    8. Relational operators in Java are used to compare two operands and determine the relationship between them. They return a boolean value (true or false). Examples: > (greater than) evaluates to true if the left operand is greater than the right; < (less than) evaluates to true if the left operand is less than the right; == (equal to) evaluates to true if both operands are equal.
    9. The primary function of logical operators in Java is to perform logical operations on boolean operands, combining or negating boolean values. The && (AND) operator returns true if and only if both of its operands are true. The || (OR) operator returns true if at least one of its operands is true.
    10. The ternary operator in Java is a shorthand if-else construct that allows you to write a conditional expression in a single line. Its syntax is condition ? value_if_true : value_if_false. For example, int min = (a < b) ? a : b; assigns the smaller of a and b to min. The benefit is more concise code for simple conditional assignments.
    11. A while loop in Java evaluates its condition before executing the loop body. If the condition is initially false, the loop body will never be executed. A do-while loop, on the other hand, evaluates its condition after executing the loop body. This guarantees that the loop body will be executed at least once, even if the condition is initially false. You might use a do-while loop when you need to perform an action at least once before checking a condition.
    12. The break statement within a switch block in Java is used to terminate the execution of the switch statement and transfer control to the statement immediately following the switch block. If a break statement is omitted, the execution will “fall through” to the next case block, even if its value does not match the switch expression.
    13. Nested if statements in Java involve placing one if statement (or if-else statement) inside another if statement (or else block). This allows for more complex decision-making based on multiple conditions. A simple scenario would be checking if a number is positive, and then, if it is, further checking if it is also even or odd.
    14. Control statements in Java are used to manage the flow of execution within a program, determining which statements are executed and in what order. Three different types of control statements discussed in the source material are if-else statements, while loops, and for loops.

    Essay Format Questions

    1. Discuss the key features of the Java programming language as outlined in the provided source. Analyze how these features contribute to Java’s popularity and suitability for various software development tasks.
    2. Explain the different categories of operators available in Java, providing examples of each type. Discuss how operators are used to manipulate data and perform computations within Java programs.
    3. Describe the various control flow statements in Java, including conditional statements and loops. Illustrate with examples how these statements enable programmers to create programs with complex logic and repetitive actions.
    4. Compare and contrast the different types of loops available in Java (while, do-while, for). Discuss the scenarios in which each type of loop is most appropriate and provide examples to support your reasoning.
    5. Analyze the concepts of exception handling in Java, including the use of try, catch, and finally blocks. Discuss the importance of exception handling for creating robust and reliable Java applications.

    Glossary of Key Terms

    • Platform Independency: The ability of a software program to run on any operating system or hardware platform without requiring significant modifications.
    • Bytecode: The intermediate code produced by the Java compiler after compiling the source code. It is platform-independent and executed by the Java Virtual Machine (JVM).
    • JRE (Java Runtime Environment): A software package that contains everything needed to run Java programs, including the JVM and supporting libraries.
    • JDK (Java Development Kit): A software development kit that includes the JRE, a compiler (javac), and other development tools needed to create, test, and debug Java applications.
    • Multi-threading: A programming technique that allows multiple parts of a program to run concurrently within a single process, improving performance and responsiveness.
    • Operator: A special symbol that performs a specific operation on one or more operands (variables or values).
    • Unary Operator: An operator that acts on a single operand (e.g., increment ++, decrement –, negation -).
    • Arithmetic Operator: An operator that performs mathematical calculations (e.g., addition +, subtraction -, multiplication *, division /, modulus %).
    • Shift Operator: An operator that shifts the bits of an operand to the left or right by a specified number of positions (<<, >>).
    • Relational Operator: An operator that compares two operands and determines the relationship between them, returning a boolean value (e.g., >, <, ==, !=, >=, <=).
    • Bitwise Operator: An operator that performs operations at the level of individual bits of the operands (e.g., & (AND), | (OR), ^ (XOR)).
    • Logical Operator: An operator that performs logical operations on boolean operands (e.g., && (AND), || (OR), ! (NOT)).
    • Ternary Operator: A conditional operator that takes three operands (condition ? value_if_true : value_if_false).
    • Assignment Operator: An operator that assigns a value to a variable (e.g., =, +=, -=, *=, /=).
    • Control Statement: A statement in a programming language that determines the flow of execution of the program (e.g., if-else, while, for, switch).
    • Conditional Statement: A control statement that allows the program to execute different blocks of code based on whether a certain condition is true or false (e.g., if, else if, else, switch).
    • Loop: A control statement that allows a block of code to be executed repeatedly until a certain condition is met (e.g., while, do-while, for).
    • break Statement: A control flow statement used to immediately terminate the execution of a loop or a switch statement.
    • Nested if Statement: An if statement that is placed inside another if statement or an else block.
    • Robust: A characteristic of a program that indicates its ability to handle errors and unexpected input gracefully, preventing crashes or incorrect behavior.
    • Exception Handling: A mechanism in programming languages to deal with runtime errors or exceptional conditions, allowing the program to recover or terminate gracefully.
    • Compile-time Error: An error that is detected by the compiler during the process of translating the source code into machine code or bytecode.
    • Runtime Error: An error that occurs during the execution of a program.
    • Garbage Collector: An automatic memory management process in Java that reclaims memory occupied by objects that are no longer in use.

    Java Programming Concepts: A Comprehensive Overview

    Document Review: Java Programming Concepts and Features

    This briefing document summarizes the main themes and important ideas and facts presented in the provided source, “002-Java_Full_Course_in_10_Hours__-03-24-2025.pdf.” The source covers fundamental aspects of the Java programming language, including its core features, operators, control statements, data structures, object-oriented principles, exception handling, file handling, threads, regular expressions, networking, and practical examples.

    **I. Core Features of Java:**

    The document begins by highlighting several key features of Java:

    * **Platform Independence:** Java follows the “Write Once, Run Anywhere” (WORA) principle. Unlike languages like C or C++, Java code is compiled into **bytecode**, which is platform-independent and can be executed on any machine with a Java Runtime Environment (JRE). “unlike other programming languages such as C or C++ Etc which are compiled into platform specific machines Java is guaranteed to be right once Run Anywhere language on compilation Java program is compiled into bite code this bite code is platform independent and can be run on any machine plus this bite code format also provides security any machine with Java runtime environment can run Java programs.”

    * **High Performance:** Although Java is an interpreted language and inherently slower than compiled languages like C++, it achieves high performance through the use of a **Just-In-Time (JIT) compiler**. “Java is an interpreted language so it will never be as fast as a compile language like C or C++ but just Java enables high performance with the use of just in time compiler so here Java provides us high performance.”

    * **Multi-threading:** Java supports multi-threading, allowing programs to perform multiple tasks concurrently by utilizing the same memory and resources. “Java multi-threading feature makes it possible to write a program that can do many tasks simultaneously the benefit of multi-threading is that it utilizes the same memory and other resources to execute multiple threats at the same time like while typing grammatical errors are checked along.”

    * **Security:** Java is considered a secure language, enabling the development of virus-free and tamper-free systems. Java programs run within the JRE with limited interaction with the host operating system. “when it comes to Security Java is always the first choice with Java secure features it enables us to develop virus free and temper free systems Java program always runs in a Java runtime environment with almost null interaction with host operating system hence it is most secure.”

    * **Portability:** The cross-platform nature of Java makes its code highly portable. “the crossplatform feature enables the Java code to be highly portable.”

    * **Object-Oriented:** Java is based on the object-oriented programming (OOP) model, where everything is treated as an object with data and behavior. This facilitates extensibility. “In Java everything is an object which has some data and behavior Java can be easily extended as it is based on objectoriented programming model.”

    * **Robustness:** Java aims to eliminate error-prone code through compile-time and runtime error checking. It features automatic garbage collection for memory management and exception handling for mishandled exceptions. “Java makes an effort to eliminate error prone codes by emphasizing mainly on compile time error checking and runtime error checking but the main areas in which Java improvised were memory management and mishandled exceptions by introducing automatic garbage collector and exception handling.”

    * **Availability:** JDK versions are available for various operating systems like Windows, Linux, and macOS. “in open jdk and the jdk versions are also available for all the types of operating systems such as Windows Linux Mac and many other flavors of Linux.”

    **II. Operators in Java:**

    The document details various types of operators in Java, explaining their functionality and providing simple code examples:

    * **Unary Operators:** Operate on a single operand (e.g., increment `++`, decrement `–`, negation `-`). Examples demonstrate pre-increment, post-increment, pre-decrement, and post-decrement.

    * **Arithmetic Operators:** Perform basic mathematical operations (+, -, \*, /, %). Examples show addition, subtraction, multiplication, division, and modulus. “Java athematic operators are used to perform addition subtraction multiplication and division they act as basic mathematic operations.”

    * **Shift Operators:** Shift bits to the left (`<<`) or right (`>>`) by a specified number of positions. An example illustrates left shift operations and how binary representation is affected. “the shift operator is used to shift the bits in the value to the left or right side of the specific number of times we have two types of shift operators which are left shift operator and the right shift operator the left shift operator moves the specific number of bits towards the left side and the right shift towards the right side.”

    * **Relational Operators:** Compare operands and return a boolean value (e.g., <, >, <=, >=, ==, !=). An example compares if `a` is less than `b` and `c`. “the Java relational operator is used to compare the operant on both the sides of the relational operator this particular operator judge whether one of the operant is greater or lesser or equal or not equal to the other operator.”

    * **Bitwise Operators:** Perform operations on individual bits of operands (&, |, ^, ~, <<, >>, >>>). An example demonstrates bitwise AND and post-increment. “basically the bitwise operator is applied on the bits for example if we provide a number to the variable that particular number will be converted into binary format and after that the operation will be applied on the bits one by one.”

    * **Logical Operators:** Combine boolean expressions (&& – AND, || – OR, ! – NOT). The relational operator example is extended to include the logical AND operator. “The Logical operators are the operators which are applied on both the ends or both the operant so the basic logical operators supported in Java programming language are and or and not the and operator is used to perform logical operation on two operant and it will result in a Boolean result so to become a true Boolean result and should be having both the operant as true and in order to result a True Value either one of the operant should be true and when it comes to not not is just a simple logical operator dat which negates the existing value that means if the value is true then after the not operation the value will be converted into false.”

    * **Ternary Operator:** A shorthand for a simple if-else conditional expression (`condition ? value_if_true : value_if_false`). An example finds the minimum of two numbers using the ternary operator. “the ternary operator is simple but highly powerful operator used in Java programming language this particular ternary operator will reduce the code length to one line.”

    * **Assignment Operators:** Assign values to variables (=, +=, -=, \*=, /=, %=, &=, ^=, |=, <<=, >>=, >>>=). An example demonstrates simple assignment and compound assignment (addition and subtraction). “the assignment operators used in Java programming language are simply used to allocate the resultant memory into the variable which is located in the left side of the operant the basic assignment operators used in Java programming language are equals to and double equals to.”

    **III. Control Statements in Java:**

    The document explains and provides examples for various control flow statements in Java:

    * **If Statement:** Executes a block of code if a specified condition is true. An example demonstrates a simple `if-else` statement to check if a number is greater than 20. “if the condition specified as true the if block will be executed otherwise the else block will be executed.”

    * **While Loop:** Repeatedly executes a block of code as long as a specified condition is true. An example prints numbers from 5 to 15 with an increment of 2. “evaluates a certain condition if the condition is true then the code is executed the process is continued until the specific condition turns out to be false the condition to be specified in while loop must be a Boolean expression.”

    * **Do-While Loop:** Similar to the `while` loop, but the loop body is executed at least once before the condition is checked. An example initializes a variable to 20, prints it once, increments it, and then checks if it’s less than or equal to 20 (resulting in a single execution). “the condition of the do V Loop is evaluated after the execution of the loop body this guarantees that the loop is executed at least once.”

    * **For Loop:** Executes a block of code a specific number of times. An example prints numbers from 1 to 10. “used to iterate and evaluate a code multiple times when the number of iterations is known by the user it is recommended to use the fall Loop.” It highlights the three parts of a `for` loop: declaration, condition, and increment/decrement.

    * **Switch Case:** Executes one block of code among many possible blocks based on the value of an expression. The document notes important points about `switch` statements, such as the use of `break` to terminate the statement sequence and the permissibility of constant case values. “used to execute a single statement for multiple conditions… certain points must be noted while using the switch state M which are one or n number of case values can be specified for a switch expression case values that are duplicate or not permissible a compile time error is generated by the compiler if unique values are not used next the case value must be a literal or a constant variables are not permissible the last condition is usage of break statement is made to terminate the statement sequence it is optional to use the statement.”

    The document then proceeds to provide practical examples of `if-else-if` (ladder), nested `if`, ternary operator usage as a conditional statement, and `switch` case statements, including a leap year checker, a car class selector, finding the greatest of three numbers, determining the greater of two numbers, and a month name printer. It also demonstrates the use of the `break` statement within a loop to terminate it prematurely based on a condition.

    **IV. Data Structures:**

    The source delves into fundamental data structures in Java:

    * **Arrays:**

    * **1D Arrays:** Explained as a multi-value container. The concept of array declaration, initialization (with a fixed size), and how arrays are stored in RAM (stack for reference, heap for the actual array with default values) is introduced. Reading and updating elements in a 1D array are demonstrated with code examples.

    * **Multidimensional Arrays (2D Arrays – Array of Arrays):** Explained as an array where each element is itself an array. The concept of different lengths for inner arrays is mentioned. Declaration, initialization, reading single elements, updating elements, and reading all elements using nested loops are illustrated with code.

    * **Strings:**

    * **String Literals (Interned Strings):** Explained how string literals are created in the “string pool” and how multiple references can point to the same literal, thus saving memory. The concept of lookup before creation and immutability is emphasized.

    * **String Objects (Using `new` keyword):** Explained how the `new` keyword always creates a new string object in the heap memory, even if the content is the same as an existing literal or object.

    * **String Methods:** A list of commonly used string methods is provided, including `equals`, `equalsIgnoreCase`, `length`, `charAt`, `toUpperCase`, `toLowerCase`, `replace`, `trim`, `contains`, `toCharArray`, `isEmpty`, `endsWith`, `startsWith`, `concat`.

    * **String API Demonstrations:** Practical code examples illustrate the usage of `length()`, `charAt()`, `toUpperCase()` (showing immutability), `toLowerCase()`, `contains()`, `substring()`, `replace()`, `toCharArray()`, and `split()` methods. A use case for validating email and phone number format using string methods is also provided. The concept of string immutability is reinforced with a concatenation example.

    * **StringBuffer:** Introduced as a mutable alternative to `String`, where data can be appended using the `append()` method without creating a new object each time.

    * **Stack:** Defined as an abstract data structure following the Last-In, First-Out (LIFO) principle. The basic operations `push` (insert) and `pop` (remove) are explained. A code example demonstrates the implementation of a stack using an array, including `push`, `pop`, `isEmpty`, `isFull`, `peek` operations, and a driver `main` method to showcase its usage.

    * **Queue:** Defined as an abstract data structure following the First-In, First-Out (FIFO) principle. The basic operations `enqueue` (insert at the rear) and `dequeue` (remove from the front) are explained. Its use in asynchronous data transfer and resource sharing is mentioned. A code example demonstrates using the built-in `Queue` interface (implemented by `LinkedList`) and its methods like `add`, `remove`, `size`, and `contains`.

    * **LinkedList:** Explained as a linear data structure where elements are linked together using nodes. The difference between `ArrayList` (array-based) and `LinkedList` (node-based) in terms of memory allocation and performance is highlighted. Various methods of the `LinkedList` class are demonstrated with code examples, including adding elements (`add`, `add(index, element)`), clearing the list (`clear()`), cloning (`clone()`), finding the index of an element (`indexOf()`), and adding elements at the end (`offer()`).

    * **HashMap:** Introduced as a map-based data structure that stores key-value pairs. Key characteristics such as allowing one null key and multiple null values, being unsynchronized, and not maintaining insertion order are mentioned. The concepts of initial capacity and load factor affecting performance are explained. Synchronized HashMap is discussed using `Collections.synchronizedMap()`, and the fail-fast behavior of iterators is noted. Common constructors are listed. A code example demonstrates basic `HashMap` operations like `put`, `get`, `containsKey`, `size`, and `clear`.

    * **ArrayList:** Described as a resizable array implementation of the `List` interface, allowing dynamic addition and removal of elements. Key features include allowing duplicate elements, maintaining insertion order, and being non-synchronized. A code example shows basic `ArrayList` operations like adding elements (`add`), getting elements (`get`), setting elements (`set`), removing elements (`remove`), checking size (`size`), iterating through elements (using a for-each loop), and sorting using `Collections.sort()`.

    * **Generics:** Explained as a feature that provides type safety to collections. The syntax for declaring generic classes and using type parameters is introduced. The benefits of generics, such as compile-time type checking and eliminating the need for manual type casting, are highlighted. An example demonstrates creating a generic `ArrayList` of strings.

    * **File Handling:** Explained as reading from and writing to files in Java using the `java.io` package. The concept of streams (byte stream and character stream) is introduced. Various useful `File` class methods are listed (e.g., `canRead()`, `canWrite()`, `createNewFile()`, `delete()`, `exists()`, `getName()`, `getAbsolutePath()`, `length()`, `list()`, `mkdir()`). Code examples demonstrate creating a new file, getting file information, writing to a file using `FileWriter`, and reading from a file using `Scanner`.

    **V. Threads in Java:**

    The document covers the fundamentals of threads in Java:

    * **Java Thread Definition:** Defined as a lightweight subprocess, the smallest independent unit of a program with a separate path of execution. Every Java program has at least one main thread.

    * **Thread Life Cycle:** The different states of a thread are explained: New, Runnable, Running, Waiting, and Terminated, with a brief description of each state.

    * **Creating Threads:** Two ways to create threads are explained:

    * **Extending the `Thread` class:** An example shows creating a class that extends `Thread` and overrides the `run()` method. Creating an instance of this class and calling the `start()` method to begin execution in a new thread is demonstrated.

    * **Implementing the `Runnable` interface:** An example shows creating a class that implements `Runnable` and implementing the `run()` method. Creating a `Thread` object by passing an instance of the `Runnable` class to the `Thread` constructor and then calling `start()` is demonstrated.

    * **Multi-threading:** Defined as the ability of a program to execute more than one thread concurrently, allowing for optimal resource utilization. A simple example shows the main thread starting two custom threads (implementing `Runnable`) and their concurrent execution. The concept of `setDaemon(true)` is briefly mentioned.

    * **Synchronization:** Briefly defined as a mechanism to control access of multiple threads to shared resources to prevent data inconsistency. An example of a scenario where synchronization is needed (booking a single movie ticket by multiple people) is given. The use of the `synchronized` keyword to acquire locks on objects is mentioned.

    * **Difference between Processes and Threads:** A table summarizes the key differences based on control, resource sharing, dependency, memory space, and management.

    * **Wrapper Classes:** Defined as classes that convert Java primitives into reference types (objects). Each primitive type has a corresponding wrapper class (e.g., `Integer` for `int`, `Boolean` for `boolean`). The concepts of boxing (primitive to object) and unboxing (object to primitive) are explained with examples. Auto-boxing and auto-unboxing are also introduced. The main reason for using wrapper classes (to treat primitives as objects) is mentioned.

    * **Keywords: `final`, `finally`, and `finalize`:**

    * **`final`:** Explained as a keyword used to make variables constant (cannot change value), classes non-inheritable, and methods non-overridable. Examples are provided for `final` variables.

    * **`finally`:** Explained as a block used in exception handling that is always executed, regardless of whether an exception is thrown or caught. Its purpose for cleanup operations (e.g., closing database connections) is highlighted.

    * **`finalize`:** Described as a protected method of the `java.lang.Object` class that is called by the garbage collector before an object is reclaimed. Its use for cleanup activities related to the object is mentioned.

    **VI. Regular Expressions (Regex):**

    The document introduces regular expressions in Java using the `java.util.regex` package:

    * **Regex Definition:** Explained as a pattern used to match a sequence of characters.

    * **`Pattern` and `Matcher` Classes:** The core classes for working with regex in Java are introduced. An example demonstrates compiling a pattern using `Pattern.compile()` and creating a `Matcher` object using `pattern.matcher()`. The `matcher.matches()` method is used to check if the entire input sequence matches the pattern.

    * **Regex Character Classes:** Various character classes are explained with their meanings (e.g., `[abc]`, `[^abc]`, `[a-zA-Z]`, `[a-d[m-p]]`, `[a-z&&[def]]`, `[a-z&&[^bc]]`, `[a-z&&[^m-p]]`). An example demonstrates using `Pattern.matches()` with different character classes to test matching.

    * **Regex Quantifiers:** Quantifiers specify the number of occurrences of a character or group. Various quantifiers are explained (e.g., `X?`, `X+`, `X*`, `X{n}`, `X{n,}`, `X{n,m}`). An example demonstrates using `Pattern.matches()` with different quantifiers to test matching.

    * **Regex Meta Characters:** Meta characters provide shortcodes for common character sets (e.g., `.`, `\d`, `\D`, `\s`, `\S`, `\w`, `\W`, `\b`, `\B`). An example demonstrates using `Pattern.matches()` with different meta characters to test matching.

    * **Practical Regex Examples:**

    * **Finding a given pattern in a string:** An example shows using `Pattern.compile()`, `Matcher`, and `matcher.find()` to locate occurrences of a specific pattern within a string.

    * **Email Validation:** An example demonstrates using a regex pattern to validate the format of an email address using `Pattern.compile()` and `matcher.matches()`.

    **VII. Networking with Sockets:**

    The document provides a basic introduction to socket programming in Java using the `java.net` package:

    * **Socket Definition:** Explained as a connection between two programs running on the same or different networks. Server sockets and client sockets are mentioned.

    * **Client-Server Communication:** A simplified explanation of how a server listens on a specific port and a client connects to the server’s IP address and port to exchange data.

    * **Basic Socket Programming Example:** Code snippets for a simple server and client are provided. The server code demonstrates creating a `ServerSocket`, accepting client connections, reading input using `BufferedReader`, and writing output using `PrintWriter`. The client code shows creating a `Socket`, connecting to the server, sending data, and receiving data. The interaction between the server and client (reading input until “over” is received) is described.

    **VIII. Who is a Java Developer?**

    The document concludes with a brief definition of a Java developer as a computer software developer or programmer who integrates Java programming language knowledge to build applications.

    **IX. Practical Java Examples:**

    The document includes several practical Java program examples:

    * **ATM Machine:** A detailed explanation and code walkthrough of a simple ATM simulation program involving account management (checking and savings), balance inquiry, deposit, and withdrawal. The program utilizes classes for `Account`, `ATM`, and `OptionMenu`, demonstrating concepts like inheritance, user input, conditional statements (`switch`), and basic transaction logic.

    * **Text Editor:** A description and code explanation of a basic text editor application built using Java Swing. It showcases features like creating/opening/saving files, editing (cut/copy/paste), font manipulation (bold, italic, size, type), color selection (foreground/background), and undo/redo functionality. It highlights the use of Swing components (`JTextArea`, `JMenuBar`, `JMenuItem`), event listeners, and the undo/redo mechanism.

    **X. Common Interview Questions and Answers:**

    A section addresses common Java interview questions:

    * **What is JVM?** Explained as the software provided by the platform that provides the runtime environment to execute Java bytecode.

    * **What is Synchronization?** Explained as a mechanism to allow controlled access of multiple threads to a shared object, preventing memory consistency errors. The `synchronized` keyword and the concept of acquiring locks are mentioned.

    * **Difference between Processes and Threads:** Summarized in a table (already mentioned in the Threads section).

    * **What is a Wrapper Class?** Explained as a way to convert Java primitives into reference types (objects). Boxing, unboxing, auto-boxing, and auto-unboxing are defined with examples. The reason for using wrapper classes (OOP adherence, passing by reference) is given.

    * **What are the keywords final, finally, and finalize?** Explained with their respective purposes and usage (already mentioned).

    * **What is OOPs?** Defined as Object-Oriented Programming System, a paradigm centered around objects. The four core principles (Encapsulation, Abstraction, Inheritance, Polymorphism) are listed and briefly explained.

    * **What is an Object?** Defined as a basic unit of OOP, an instance of a class with state (data) and behavior (methods). The process of object creation in stack (reference variable) and heap (actual object) memory is explained.

    * **Difference between Stack and Heap Memory:** Key differences are summarized based on usage, accessibility, memory management (LIFO vs. generational/key-value), and lifetime.

    * **What are the different access modifiers in Java?** Four access modifiers (`public`, `private`, `protected`, and default/package-private) are listed and their visibility/accessibility scopes are explained.

    * **Difference between HashTable and HashMap:** Key differences are highlighted based on null keys/values, synchronization, performance, and inheritance.

    * **Difference between equals() and == operator:** Explained with a demonstration of comparing String objects. `==` compares object references (memory addresses), while `equals()` (in the case of `String`) compares the actual content of the objects.

    * **Predict the Output of a Java Program:** A simple program with method calls and arithmetic operations is given, and the expected output (11) is explained step-by-step.

    In summary, the provided source offers a comprehensive overview of fundamental Java programming concepts, supported by explanations and practical code examples. It covers essential language features, data structures, object-oriented principles, and common programming tasks, making it a valuable resource for learning Java.

    Java Loop Control Statements: for, while, do-while

    Java supports several types of loop control statements that allow you to execute a block of code repeatedly until a certain condition is met. The main types of loops in Java discussed in the sources are the for loop, the while loop, and the do while loop.

    1. for Loop:

    • The for loop in Java is used to iterate and execute a block of code multiple times when the number of iterations is known by the user.
    • It is recommended to use the for loop when the number of iterations is known.
    • A for loop has three basic parts:
    • Initialization: This step initializes the loop counter variable. For example, int i = 1.
    • Condition: This part specifies the condition that is evaluated before each iteration of the loop. The loop continues to execute as long as this condition is true. For example, i <= 10.
    • Increment or Decrement: After each iteration of the loop, the counter variable is updated. For example, i++.
    • Example:for (int i = 1; i <= 10; i++) {
    • System.out.println(i);
    • }
    • This for loop will print the value of i ten times, from 1 to 10.
    • A flow diagram of a for loop shows that initialization happens first, then the condition is checked. If true, the code inside the loop executes, followed by the increment/decrement, and the condition is checked again. If the condition is false, the loop terminates.

    2. Enhanced for Loop (or Advanced for Loop):

    • This type of for loop is used to iterate over elements in arrays and collections.
    • It simplifies the process of iterating through each element without needing to manage indices explicitly.
    • The syntax typically involves declaring a variable of the same type as the elements in the array/collection, followed by a colon, and then the array/collection itself.
    • Example:String[] arrData = {“John”, “Jerry”, “Ralph”, “Jim”, “Tom”};
    • for (String name : arrData) {
    • System.out.println(name);
    • }
    • This loop will iterate through the arrData array and print each name.
    • The flow of an enhanced for loop involves checking if there are any elements present in the array/collection. If true, it initializes a local variable with the first element and executes the loop body. This process continues for each element until all elements have been processed.

    3. while Loop:

    • The while loop evaluates a certain condition.
    • If the condition is true, the code inside the while loop is executed.
    • This process continues until the specified condition becomes false.
    • The condition to be specified in a while loop must be a Boolean expression.
    • Example:int i = 5;
    • while (i <= 15) {
    • System.out.println(i);
    • i += 2;
    • }
    • This while loop will print values of i starting from 5, incrementing by 2, as long as i is less than or equal to 15.
    • A flowchart of a while loop shows that the condition is checked first. If true, the code inside the loop executes, and then the condition is checked again. If the condition is false, the loop terminates.

    4. do while Loop:

    • The do while loop is similar to the while loop, but with a key difference: the condition is evaluated after the execution of the loop body.
    • This guarantees that the loop is executed at least once, regardless of whether the condition is initially true or false.
    • Example:int i = 20;
    • do {
    • System.out.println(i);
    • i++;
    • } while (i <= 20);
    • In this do while loop, the value 20 will be printed once, even though the condition i <= 20 becomes false after the increment.
    • A flow diagram of a do while loop shows that the statements inside the loop are executed first, and then the condition is checked. If the condition is true, the loop repeats; otherwise, it terminates.

    5. Nested Loops:

    • It is possible to have one loop inside another loop; this is known as a nested loop.
    • For each iteration of the outer loop, the inner loop executes completely.
    • Example:int[][] array = {{1, 2}, {3, 4}};
    • for (int i = 0; i < array.length; i++) {
    • for (int j = 0; j < array[i].length; j++) {
    • System.out.println(array[i][j]);
    • }
    • }
    • This example demonstrates a nested for loop iterating through a two-dimensional array. The outer loop iterates through the rows, and the inner loop iterates through the elements in each row.

    6. break Statement in Loops:

    • The break statement can be used to terminate a loop prematurely.
    • When a break statement is encountered inside a loop, the control flow immediately exits the loop and continues with the statement immediately following the loop.
    • Example:int[] numbers = {10, 20, 30, 40, 50};
    • for (int num : numbers) {
    • System.out.println(num);
    • if (num == 40) {
    • break;
    • }
    • }
    • In this example, the loop will print 10, 20, and 30, and then when num is 40, the break statement will be executed, and the loop will terminate before printing 50.

    7. Infinite Loop:

    • An infinite loop occurs when the condition of the loop never becomes false, causing the loop to execute indefinitely unless it is terminated externally or by a break statement.
    • A common way to create an infinite loop with a while loop is to use the condition true.
    • Example:while (true) {
    • System.out.println(“Edureka”);
    • // Some condition to break out of the loop should be present
    • // to avoid truly infinite execution.
    • }
    • This while loop will continuously print “Edureka”.

    The source also briefly mentions the if statement as a control statement, but it is a conditional statement, not a loop, as it does not involve repeated execution of code based on a condition. Similarly, the switch case is a selection statement for executing a single block of code among multiple cases.

    Java Switch Case Control Statement

    The switch case statement in Java is a control statement used to execute a single statement for multiple conditions. It provides a way to choose one block of code to execute from a set of possible cases based on the value of an expression.

    Here are some key aspects of the switch case statement based on the sources:

    • Purpose: To execute a single statement among multiple conditions.
    • Supported Data Types: The switch case statement can be used with short, byte, int, long, enum types, Character, String, and their corresponding wrapper types.
    • Syntax: A switch statement includes a switch expression and one or more case labels. The switch expression is evaluated once, and its value is compared with the values of each case label.
    • switch (expression) {
    • case value1:
    • // statements to be executed if expression equals value1
    • break;
    • case value2:
    • // statements to be executed if expression equals value2
    • break;
    • // …
    • default:
    • // statements to be executed if expression does not match any case
    • }
    • Rules for Usage:
    • One or n number of case values can be specified for a switch expression.
    • Case values must be unique. Duplicate case values are not permissible and will result in a compile-time error.
    • Case values must be literals or constants. Variables are not permissible as case values.
    • The break statement is used to terminate the statement sequence within a case. If a break statement is not used, the execution will “fall through” to the next case. The use of the break statement is optional.
    • There can be at most one default label. The default block is executed if the value of the switch expression does not match any of the case values. The default label is also optional and does not need a break statement after it.
    • Example: The source provides an example where an instrument number is used in a switch statement to print the name of the selected instrument:
    • int instrument = 4;
    • switch (instrument) {
    • case 1:
    • System.out.println(“piano”);
    • break;
    • case 2:
    • System.out.println(“drums”);
    • break;
    • case 3:
    • System.out.println(“guitar”);
    • break;
    • case 4:
    • System.out.println(“flute”);
    • break;
    • case 5:
    • System.out.println(“uka”);
    • break;
    • case 6:
    • System.out.println(“volin”);
    • break;
    • default:
    • System.out.println(“invalid”);
    • }
    • In this example, because instrument is 4, the output will be “flute”.

    In our previous conversation, we discussed various control statements in Java, including loops. The switch case statement is another important control statement that allows for conditional execution based on the value of an expression, offering an alternative to multiple if-else-if statements in certain situations. The source also mentions switch as one of the types of conditional statements available in Java.

    Java Object-Oriented Programming Fundamentals

    Object-Oriented Programming (OOP) is described as a programming style associated with programming concepts such as inheritance, polymorphism, abstraction, and encapsulation. Java is highlighted as one of the most sought-after skills and a language that follows an object-oriented programming paradigm. Whether you are a beginner or an experienced professional, understanding these key OOP concepts is crucial for building modular and maintainable code in Java.

    At its core, OOP is a methodology or a paradigm using which we can design our software solutions. It centers around the concepts of objects and classes. An object is a real-world entity, anything you can see, touch, or feel, and it has both state (properties or attributes) and behavior (methods). A class acts as a blueprint or a drawing of an object, defining how an object will look and behave through its properties and behaviors.

    The source emphasizes that having an object-oriented programming approach allows for creating different types of objects that share the same properties but have different values for those properties. OOP is presented as a bottom-up approach where you first think of objects and then start coding, contrasting with the top-down approach of procedural programming. It also offers better security through access modifiers compared to procedural approaches where data moves freely. Furthermore, OOP provides features like overloading, overriding, and inheritance that facilitate easier software design.

    The source identifies four major fundamentals or pillars of OOP:

    • Encapsulation: This is a mechanism where you bind your data and code together as a single unit. It also means to hide your data in order to make it safe from any modification. The analogy of a medical capsule is used, where the drug (data) is safe inside the capsule. Encapsulation ensures that the methods and variables of a class are hidden and protected. An example is given where user details like name, account number, email, and balance are encapsulated within a class.
    • Inheritance: This is where the property of an object will be acquired by the other object. It establishes an “is a” relationship, like a parent-child relationship. A child class (sub class) inherits properties and methods from a parent class (super class). The source discusses different types of inheritance in Java:
    • Single-level inheritance: One parent class and one child class.
    • Multi-level inheritance: A chain of inheritance, like parent to child to grandchild.
    • Hierarchical inheritance: Multiple child classes inheriting from a single parent class.
    • The source explicitly states that multiple inheritance or hybrid inheritance is not directly supported in Java; interfaces are used to achieve similar functionality. Inheritance promotes code reusability and extensibility, and allows for overriding methods in the child class to provide specific implementations. It can also be used for data hiding by making attributes private in the parent class.
    • Abstraction: This “refers to the quality of dealing with ideas rather than events”. It basically deals with hiding the details and showing the essential things to the user. When you make a call, you only see the option to pick up or reject, without knowing the internal processing. Abstraction helps to reduce code complexity. It can be achieved in two ways in Java:
    • Abstract classes: A class declared with the abstract keyword. An abstract class cannot be instantiated (you cannot create an object of it). It can contain abstract as well as concrete methods. An abstract class can act as a template for its subclasses. The runtime environment can construct the parent object of an abstract class before constructing the child object.
    • Interfaces: Known as a “blueprint of a class” or a collection of abstract methods and static constants. Each method in an interface is public and abstract and does not contain any constructor. Interfaces also help in achieving multiple inheritance in Java. A class implements an interface using the implements keyword.
    • Polymorphism: Meaning “taking many forms” (poly – many, morph – forms). It is “the ability of a variable, function, or an object to take on multiple forms“. Polymorphism allows you to define one interface or method and have multiple implementations. Java supports two types of polymorphism:
    • Runtime Polymorphism (Dynamic Polymorphism): A call to an overridden method is resolved at run time rather than at compile time. Method overriding is an example of runtime polymorphism, where a child class provides a specific implementation of a method already defined in its superclass. Upcasting (reference variable of the parent pointing to the object of the child) is a key aspect of runtime polymorphism.
    • Compile-time Polymorphism (Static Polymorphism): A call to an overloaded method is resolved at compile time rather than at run time. Method overloading is an example, where a class has two or more methods with the same name but different arguments (number, types, or order of parameters).

    In summary, Object-Oriented Programming in Java provides a powerful and structured approach to software development by organizing code around objects and classes and leveraging the principles of encapsulation, inheritance, abstraction, and polymorphism to create robust, modular, reusable, and maintainable applications.

    Java Inheritance: Concepts and Types

    Java Inheritance is a fundamental concept of Object-Oriented Programming (OOP) in Java where a child class (also known as a subclass or derived class) inherits the methods and properties of a parent class (also known as a superclass or base class). The source explains that inheritance establishes an “is-a” relationship between the classes, like the relationship between a parent and a child. For example, a Mobile is a Product.

    Here are the key aspects of Java Inheritance discussed in the sources:

    • Mechanism: Inheritance is achieved in Java using the extends keyword. When a class extends another class, it gains access to the non-private members (fields and methods) of the parent class.
    • Benefits:
    • Code Reusability: Inheritance promotes code reusability by allowing the child class to reuse the code already written in the parent class. This reduces redundancy and makes the code more maintainable.
    • Extensibility: Inheritance enables you to add new features or modify existing ones in the child class without altering the parent class.
    • Method Overriding: A child class can override (redefine) a method inherited from the parent class to provide its own specific implementation. This is a key aspect of achieving runtime polymorphism in Java.
    • Data Hiding: By declaring attributes as private in the parent class, you can control their accessibility in the child class, contributing to data hiding.
    • Types of Inheritance Supported by Java: The source outlines four types of inheritance in Java:
    • Single Inheritance: One child class inherits from one parent class. The source provides an example where a HadoopTeacher class inherits from a Teacher class.
    • Multi-level Inheritance: A child class inherits from a parent class, which in turn inherits from another parent class, forming a chain of inheritance. The source illustrates this with Class B inheriting from Class A, and Class C inheriting from Class B.
    • Hierarchical Inheritance: Multiple child classes inherit from a single parent class. The source gives an example where Class B and Class C both inherit from Class A. Another example involves a Restaurant acting as a parent class with child classes like ItalianRestaurant, ChineseRestaurant, and MexicanRestaurant inheriting from it.
    • Hybrid Inheritance: This is a combination of multiple inheritance and multi-level inheritance. The source mentions that while direct multiple inheritance (one class inheriting from multiple classes) is not supported in Java due to the “diamond problem,” hybrid inheritance can be achieved through the use of interfaces.
    • Multiple Inheritance: The source explicitly states that multiple inheritance is not directly supported in Java. This is to avoid the ambiguity arising from inheriting methods with the same signature from multiple parent classes (the “diamond problem”). Java uses interfaces to achieve a form of multiple inheritance by allowing a class to implement multiple interfaces.
    • “Is-a” and “Has-a” Relationships: The source also briefly touches upon “is-a” and “has-a” relationships. Inheritance represents an “is-a” relationship (e.g., BMW is a Car). A “has-a” relationship, on the other hand, involves a class having an instance of another class as a member (e.g., a BMW has an Engine). Inheritance is implemented using extends, while the “has-a” relationship is typically implemented through instance variables.

    In our previous discussion about OOP, we established that inheritance is one of the four pillars, enabling code reuse and forming hierarchical relationships between classes. The new source provides more detailed explanations and examples of the different types of inheritance in Java and clarifies the language’s approach to the complexities of multiple inheritance through interfaces.

    Java Exception Handling: A Comprehensive Overview

    Exception Handling in Java is a crucial mechanism to deal with unwanted or unexpected events that occur during the execution of a program at runtime and can disrupt the normal flow of the program instructions. The source emphasizes that if exceptions are not handled, it can lead to system failure. Java’s exception handling mechanism aims to manage these runtime errors so that the normal execution flow is not disrupted.

    Here’s a breakdown of exception handling in Java as discussed in the sources:

    • What is an Exception? An exception is an unwanted or unexpected event that occurs during program execution. It’s a problem that arises during the execution of a program and can happen for various reasons like invalid user input, a file not being found, network connection loss, or the JVM running out of memory.
    • Error vs. Exception: The source distinguishes between errors and exceptions:
    • An error indicates a serious problem that a reasonable application should not try to catch. Errors are typically impossible to recover from and are usually related to the runtime environment, such as virtual machine error or stack overflow error.
    • An exception indicates conditions that are reasonable for an application to try to catch. Exceptions can be recovered by handling them. They can be caused by the application itself (e.g., NullPointerException) or external factors (e.g., IOException).
    • How JVM Handles Exceptions: When an exception occurs within a method, the JVM:
    • Creates an exception object.
    • This object contains the name and description of the exception and the current state of the program where the exception occurred.
    • Throws this exception object to the runtime system (JVM).
    • The JVM then tries to find an appropriate exception handler to deal with the exception.
    • Exception Hierarchy: All exception and error types in Java are subclasses of the Throwable class, which is the base class of the hierarchy. Throwable has two main branches:
    • Exception: This class is used for exceptional conditions that user programs should catch. Examples include NullPointerException, RuntimeException, IOException, SQLException, ClassNotFoundException, and ArrayIndexOutOfBoundsException.
    • Error: This branch is used by the Java runtime system to indicate errors related to the runtime environment itself (JRE). Examples include VirtualMachineError and StackOverflowError.
    • Checked vs. Unchecked Exceptions: Exceptions in Java are categorized into checked and unchecked:
    • Checked Exceptions: These are exceptions that the compiler forces you to handle (using try-catch or declaring them using throws) because they are considered potential issues that might occur during normal operation (e.g., IOException, SQLException). They happen at compile time.
    • Unchecked Exceptions (Runtime Exceptions): These exceptions are not checked by the compiler, and you are not forced to handle them (e.g., ArithmeticException, ArrayIndexOutOfBoundsException, NullPointerException). They typically result from programming errors and happen at runtime.
    • Exception Handling Keywords: Java provides specific keywords to handle exceptions:
    • try: The try block encloses the code that might throw an exception.
    • catch: The catch block follows the try block and contains the code that handles a specific type of exception. There can be multiple catch blocks to handle different types of exceptions (multi-catch).
    • finally: The finally block follows the try (and optional catch) block and contains code that will always be executed, regardless of whether an exception was thrown or caught. This is typically used for cleanup operations like closing connections. The catch block is not required if a finally block is present.
    • throw: The throw keyword is used to explicitly throw an exception. You can throw built-in exceptions or create and throw your own user-defined exceptions.
    • throws: The throws keyword is used in the method signature to declare that a method might throw a certain type of exception. It doesn’t throw the exception itself but indicates that the calling code needs to handle it.
    • Nested try Blocks: A try block can be placed inside another try block, creating nested try blocks. This allows for handling exceptions in different scopes.
    • User-Defined Exceptions: Java allows you to create your own custom exception classes by extending the Exception class (or its subclasses). This is useful when the built-in exceptions don’t adequately describe a specific error situation in your application. User-defined exceptions are thrown using the throw keyword.
    • Passing Exceptions: The source provides an example of how exceptions can occur during operations like date format conversion and how these exceptions can be caught and handled.
    • Methods of Exception Handling: The source lists try, catch, finally, throw, and throws as the various methods of exception handling in Java.
    • Difference Between throw and throws:
    • throw is used to explicitly throw an exception, while throws is used to declare an exception that a method might throw.
    • throw is followed by an instance of an exception, while throws is followed by a class name.
    • throw is used within a method’s body, while throws is used with the method signature.
    • throw can throw only one exception at a time, while throws can declare multiple exceptions.
    • Difference Between final, finally, and finalize: These are distinct keywords with different purposes:
    • final: Used to apply restrictions on classes (cannot be inherited), methods (cannot be overridden), and variables (cannot be changed).
    • finally: A block used to execute important code regardless of whether an exception is handled or not.
    • finalize: A method used to perform cleanup processing just before an object is garbage collected.

    Previously, we discussed how exception handling is a key feature that sets Java apart as a programming language. This new source provides a comprehensive overview of the concepts and mechanisms involved in handling exceptions in Java, which is crucial for building robust and fault-tolerant applications.

    Java Full Course in 10 Hours | Java Tutorial for Beginners [2025] | Java Online Training | Edureka

    The Original Text

    hello everyone and welcome to this Java full course by urea Java stands as one of the most influential programming languages powering everything from mobile applications to large scale Enterprise systems over the years Java has become a preferred choice for developers due to its platform independence strong security features and object oriented approach from Android apps and web appli ations to cloud-based Solutions and financial systems Java is everywhere whether you are an absolute beginner looking to learn your first programming language or a season professional aiming to refine your Java expertise this course is designed to help you understand key concepts with practical examples and hands- on learning so now let us outline the agenda for this Java full course in this course We Begin by answering the fundamental question such as what is Java I’m guiding you through the Java installation process to get you started on the right foot we will cover Core Concepts such as operators in Java and conditional statements in Java which lay the ground work for more advanced programming next we will dive into Java oops Concepts exploring the principle of objectoriented programming that make Java a powerful tool for building modular and maintainable code we will walk you through abstract classes inheritance and polymorphism providing clear examples and Hands-On exercises to solidify your understanding as we progress we will tackle exception handling in Java to ensure your programs are robust and error resistant you will learn how to work with arrays and navigate the collections Frameworks which are essential for efficient data management the course will also explore Java strings and interfaces setting the stage for more complex data structures we then move into specialized topics such as data structures in Java including in-depth discussion on link list and hashmap as well as gener to enhance type safety in your programs file handling Java threats and regular EXP option in Java will expand your toolkit for developing Advanced applications while soit programming in Java opens up the world of network communications finally we will guide you on how to become a Java developer showcase real well Java projects and prepare you for the essential Java interview question and answers to help you land your dream job in the tech industry but before we jump into the course content please like share and subscribe to our YouTube channel and hit the Bell icon to stay updated on the latest day content from edura also edure rea’s Java certification training crafted by top industry experts is designed to help you become an oracle certified professional this Java course covers skills such as objectoriented programming Java collection s SOA and web services also working with jdbc design patterns and along with popular development tools so check out the course Link in the description box below now let’s get started with our first topic what is Java there are many controversies that say Java is going to die and it has become weak compared to its competitors so many beginners and experienced programmers have a doubt if it’s really dying to clear this out let us have a quick look on the major applications and websites once ABNB Uber Facebook Instagram Google Amazon Netflix and lot many we shall begin with little basic history of java now why did we recall all these applications and websites it is because all these sck Giants are in the use of none other than Java programming language it is the base of all data frames storage and parts of every single major application this entirely proves that Java is never going to die it is getting evolved every single day and it is one of the major reasons behind the development of much major softwares now that we have a good clarification let us move ahead and learn Java really fast Java is also used in multiple ways for example social media big data and Hadoop Android application development data science and artificial intelligence data security software testing and data analytics now that we have a good clarification let us move ahead and learn Java really fast we shall begin with a little basic and history of java firstly history of Java Java was invented by James Gosling in the year 1995 it was basically invented for interactive television for God’s sake it was found too advanced and was also able to perform extraordinary and sizable tasks than just a television thing now let’s get started with Java installation go to your Chrome browser then search Java download hit enter go to the second website because we want the Java from Oracle when you click on that link you’ll be redirected to a page that will look something like this here it says jdk 21 is latest long-term support release so we want jdk 22 and below that you can see that it is for different machines so if you want for Linux you can go ahead and directly download it and if you want for Mac OS you have to change it and same goes for Windows now in Windows also you can download it using compressed archive or MSI installer but we’ll go ahead with the normal installer so download has started I’ll just go to my download folder and let’s see so here is our jdk file now let’s go ahead and run it it will ask you for the permission after that just click on next and I’ll just go ahead with the default it is best to keep it default now it will install after that just close it to verify this we’ll just go ahead and type CMD and type Java Das Dash version now we have installed jdk now we’ll just download one of the IDS the most popular in the market is Eclipse so for that just go ahead and type Eclipse download you can see the first site just go ahead and click it after that you’ll see page that will look something like this here’s the download option you can go ahead and click on this it will redirect you to this page now latest one is 20246 so this is for the June 2024 you can directly install eclipse in your system or you can download the specific package now we are going to go ahead with Java so this is for our Java and we are using window so click on this one after that click on download let me go to the download folder let’s see now we can see this is there and after that just extract it here now let’s open this folder here you can see the eclipse application just click on that it might take a while after that you can see this I’ll just keep the default and click on launch now it is starting Eclipse you might see a welcome page that you have to close now I want to create a Java project so just going to go to files and new ones now here I’ll just go to others and find Java click on Java project and then next after this let’s name our new project my first project make sure you don’t have space in the name after that we are going to use Java ac22 after that configure J then click on ADD now make sure you have selected standard VM standard virtual machine click on next and just click on the directory go to your local disk then into the program files there you can see your Java folder just go ahead and click on that here there are two options jdk 21 and 22 now we’re going to go ahead with 22 and just select the folder now since jdk 22 came here I’ll just go ahead and finish it after that you can see jdk 22 is here just apply it and apply and close after that nothing just finish it so so we can see the my first project is here now I want to create a class main class so I’ll just go ahead and click on file and new and here we can see class click on that after that make sure you have used public and then clicked on public static void main string arguments I’ll name this default the package name should be there and the name I’ll go ahead with me after that just go ahead and click on finish now going see the main Java file is here so this is our first program now let’s go ahead and run our favorite hello world program make sure you are typing correctly and don’t forget to use semicolon at the end now just go ahead and click on run over here it will ask you that you want to use yes so this is the console where you’ll get your output and since we havep print L and hello world we get the hello world now let’s try something else I’ll copy this and then paste it again now this time I’ll do please subscribe now let’s run it again it will start compiling again and if you don’t want to get annoyed by it just click always allow and now you know how to install Java and also if we have tried a simple code using it some of the major features are as follows firstly open source Java since its birth till today is an open source product there are many speculations that say Java is not free anymore but to be sure Java is still free and the open jdk and Oracle jdk are completely similar to each other other that means you can typee your code using Oracle jdk and execute it in open jdk and the jdk versions are also available for all the types of operating systems such as Windows Linux Mac and many other flavors of Linux followed by the first feature the next feature is high performance Java is an interpreted language so it will never be as fast as a compile language like C or C++ but just Java enables high performance with the use of just in time compiler so here Java provides us high performance followed by that the next feature is multi-threading Java multi-threading feature makes it possible to write a program that can do many tasks simultaneously the benefit of multi-threading is that it utilizes the same memory and other resources to execute multiple threats at the same time like while typing grammatical errors are checked along followed by this feature the fourth feature is secure when it comes to Security Java is always the first choice with Java secure features it enables us to develop virus free and temper free systems Java program always runs in a Java runtime environment with almost null interaction with host operating system hence it is most secure followed by this the next feature is Java’s platform independency unlike other programming languages such as C or C++ Etc which are compiled into platform specific machines Java is guaranteed to be right once Run Anywhere language on compilation Java program is compiled into bite code this bite code is platform independent and can be run on any machine plus this bite code format also provides security any machine with Java runtime environment can run Java programs followed by this the next feature is portability the crossplatform feature enables the Java code to be highly portable in Java everything is an object which has some data and behavior Java can be easily extended as it is based on objectoriented programming model the last but not the least feature of java is its robust nature Java makes an effort to eliminate error prone codes by emphasizing mainly on compile time error checking and runtime error checking but the main areas in which Java improvised were memory management and mishandled exceptions by introducing automatic garbage collector and exception handling we have previously examed Java’s key features and what sets it apart as a programming language in the operators in Java Java module Learners will learn about the various operators available in Java and how to use them to perform different types of operations within their code the operators in a programming language are specific or special symbols used along with the variables on numbers so as to carry out some specific operations the various operators available in Java are as follows there un Operator athematic Operator shift operator relational op Operator bitwise Operator logical Operator ternary Operator and finally the assignment operator we shall discuss about each and every single operator available in Java firstly the unary operator the Java Unity operator requires only one operant the UN operators are used to perform various operations such as incrementing or decrementing a value by one negating an expression inverting the value of aoan and many more now let us execute a simple example of unary operator in Java as you can see this particular example is based on unary operators so here I have declared a value 10 to the variable X of integer data type and now I’m trying to increment the value of x by 1 by adding the unary operator increment this particular incrementation is called post increment followed by that we have a similar increment operator which is placed before X which says it is a pre-increment operator followed by that we will try to decrement the value of x by1 by using post decrement operator where we will provide the decrement operator after X and similarly the same decrement operator is used before X when which says that it is a pred decrement operation now let us try to execute our program and see the output you can see the output has been successfully generated now followed by this let us move ahead and understand the athematic operator Java athematic operators are used to perform addition subtraction multiplication and division they act as basic mathematic operations now let us see a basic example for athematic operators you can see I have provided two variables that is a and b a has the value 10 and B has the value 5 so the basic arithmatic operations that I will be performing are addition subtraction multiplication division and modulus so the addition symbol is carried out by plus subtraction is minus is multiplication is star division is forward slash and modulus is modulus symbol these are common now let us execute this and see the output you can see all the athematic operations have been successfully executed and the output is also generated so you can see the addition is 15 subtraction is 5 multiplication is 50 division is 2 and modulus S zero now with this let us move ahead into the next type of operators which are none other than the shift operators the shift operator is used to shift the bits in the value to the left or right side of the specific number of times we have two types of shift operators which are left shift operator and the right shift operator the left shift operator moves the specific number of bits towards the left side and the right shift towards the right side now let us execute a sample program to understand the shift operators in a much better way you can see I have used shift operator here which is left shift and I’m using left shift operator to move two bits of 10 towards left side and three bits of 10 towards left side and two bits of 20 towards left side and four bits of 15 towards left side so basically what happens here is 10 will be converted into to binary numbers and similarly all the other numbers will be converted into binary numbers and the number of bits which we require that is two will be shifted in that particular converted binary number and the new number generated will be provided in decimal numbers and that will be our output let us execute this program and see our output you can see that the program has been getting executed and the new numbers are been generated here as the output so 10 after shifting two bits towards left will be generating 40 and 10 after shifting three bits towards left will generate 80 and similarly 15 after shifting four bits will generate in 240 now that we have understood the shift Operator Let us move ahead and understand the relational operator the Java relational operator is used to compare the operant on both the sides of the relational operator this particular operator judge whether one of the operant is greater or lesser or equal or not equal to the other operator now the basic operation what a relational operator does is it compares the two numbers for example if you have a as five and B as 10 and if you compare if a is greater than b then the result will be false because a is smaller than b so this is what a relational operator basically does now let us go through a sample program to understand relational operators in a much better way here you can see that I have declared three variables that is a b and c a has 10 B has 5 and C has 20 now what we doing is we comparing a is less than b and a is less than C and followed by that we’re comparing again a is less than b and a is less than C so this example is both for relational operator as well as logical operator so The Logical operation which we have used is and now let us try to execute this program and see the output and there you go the result is false because a is greater than b but not less than b similarly a is less than C but the result of and logical operator is false because to become true both sides of the operant should be true now let us move ahead and understand bitwise operator basically the bitwise operator is applied on the bits for example if we provide a number to the variable that particular number will be converted into binary format and after that the operation will be applied on the bits one by one let us execute a sample program to understand bitwise operators in a much better way so this particular example is based on bitwise operators here also we have the same three values that is a = to 10 B is = to 5 and C isal to 20 now we’re performing bitwise operations on the particular variables and comparing them using a logical operator which is and now let us try to execute this program and see the output you can see that the program has been successfully executed and the output is been printed but the value of a is been changed here from 10 to 11 this happened because it is based on the bitwise operation which we performed here that is A++ post increment now followed by this let us move ahead and understand the next type of operators that are The Logical operators The Logical operators are the operators which are applied on both the ends or both the operant so the basic logical operators supported in Java programming language are and or and not the and operator is used to perform logical operation on two operant and it will result in a Boolean result so to become a true Boolean result and should be having both the operant as true and in order to result a True Value either one of the operant should be true and when it comes to not not is just a simple logical operator dat which negates the existing value that means if the value is true then after the not operation the value will be converted into false now let us see one basic example to understand logical operators in a much better way you can see the previous example is the best fit to understand logical operators you can see the value here was false since a is greater than b so this value will result in a false and this value will result in false so false and false will result in false now moving ahead we shall understand the next type of operators which are the ternary operators the ternary operator is simple but highly powerful operator used in Java programming language this particular ternary operator will reduce the code length to one line to understand this in a much better way let us execute a sample program here you can see that we have two different values which are A and B A has 2 and B has five so the ternary operator is this particular statement here we have also allocated a new variable which is of integer type which is minimum so minimum is equals to a less than b if a is less than b then minimum is a and else B is equals to minimum so this particular operation is minimum here a is compared if a is truly less than b then a will be printed into minimum else B will be printed into minimum and finally we shall print the value which is stored into minimum now let us execute this and see the output you can see that the value of a which is two is been printed here after executing the ternary operator present in the line number seven now let us move ahead and understand the next type of operator which are the assignment operators the assignment operators used in Java programming language are simply used to allocate the resultant memory into the variable which is located in the left side of the operant the basic assignment operators used in Java programming language are equals to and double equals to to understand assignment operators in a much better way let us execute a sample Java program so this particular program is based on assignment operations here we have two different values which are A and B A isal to 10 and B is = to 20 now what we are basically doing here is using the assignment operator and adding value 4 to the value 10 and decrementing value 4 to the value 20 this particular operator used here is the assignment operator now the result will be AAL to 14 and B is equal to 16 now let us execute this program and see if the output will be correct or not you can see the output is as expected with this let us move ahead and understand the control statements in Java a control statement in Java is a statement that determines whether the other statement will be executed or not in simple terms it controls the flow of a particular program in Java now let us see the control statements which are present in in Java so the control statements supported in Java are if else Loop while loop do while loop for Loop and switch case so these are the basic control statements that each and every programmer must be known with now let us understand each and every one of these control statements and also execute a basic program to understand them in a much better way firstly we shall deal with the if statement or if control statement in this statement if the condition specified as true the if block will be executed otherwise the else block will be executed we shall execute a sample program to understand the IFL statement in a much better way so this particular program is based on if else control statement here we have the input a equal to 15 the first if condition will be in case if a is greater than 20 then the first print of statement should be executed else the other print of statement should be executed now let us execute this program and see the output since we know 15 is less than 20 we should be expecting the second set of statements to be executed which is a is less than 10 and hello world you can see the message has been successfully printed here followed by this let us move ahead and understand the next type of control statements which are supported in Java that is none other than the V Loop known as the most common Loop the V Loop evaluates a certain condition if the condition is true then the code is executed the process is continued until the specific condition turns out to be false the condition to be specified in while loop must be a Boolean expression an error will be generated if the type is used is not an integer type or a string type let us execute a basic example to understand this while loop in a much better way as you can see this particular example is based on while loop here we have a variable I which has the value five here the condition specified in by Loop is I should be less than or equal to 15 until then we have to execute the print of statement which prints the value of I and after that the value of I will be incremented by two now this particular set of Loop will be executed until the value of I is less than or equal to 15 once after this condition becomes true the loop will be terminated let us try to execute this program and see the output you can see the values 5 7 9 11 13 and 15 are printed successfully once after the value of I reached to 15 the loop got terminated and it exited the loop now followed by this let us understand the next type of loop which is the do while loop the do while loop is completely similar to The V Loop the only difference is that the condition of the do V Loop is evaluated after the execution of the loop body this guarantees that the loop is executed at least once now let us try to execute a sample program to understand do V Loop in a much better way you can see that this particular example is based on dowi Loop here we have a variable I which stores the value 20 so once after the value is been declared we have the Dov Loop so inside the Dov Loop we are supposed to print the value of I the value of I will be incremented by one every single time the loop is executed now the loop body of duy loop is finished so the condition is placed at the end of DUI Loop which says the the I value should be less than or equal to 20 until then this particular Loop is valid now the only suspense between du V Loop and V Loop is the do V Loop will be executed at least for once without checking the condition because the condition is placed after the do Loop body now let us try to execute this and see the output you can see that the Loop has been successfully executed the value 20 is printed only for once because the value of I which we declared was 20 and it is true according to the condition so the control got terminated let us move ahead and understand the next type of loop which is the fall Loop the fall Loop in Java is used to iterate and evaluate a code multiple times when the number of iterations is known by the user it is recommended to use the fall Loop to understand the fall Loop in a much better way let us try to execute one basic example based on Fall Loop so this particular example is based on for Loop here we have the value of i as 1 and the condition is I is less than or equal to 10 and after that a post increment so in this particular statement we will be printing the value of I 10 times until the for Loop is true the three basic parts of a follow Loop are declaration condition and increment or decrement so this particular first block which is I is equal to 1 is the Declaration part and after that I is less than or equal to 10 is the condition part and lastly i++ which is the increment is the last part now let us try to execute this program and see the output you can see that the program has been successfully executed and the value of I is printed for 10 times which is 1 2 3 4 5 6 7 8 and 9 and 10 now followed by this let us understand the next type of loop which is the switch case a switch statement in Java is used to execute a single statement for multiple conditions the switch case statement can be used with short bite int long enum types Etc certain points must be noted while using the switch state M which are one or n number of case values can be specified for a switch expression case values that are duplicate or not permissible a compile time error is generated by the compiler if unique values are not used next the case value must be a literal or a constant variables are not permissible the last condition is usage of break statement is made to terminate the statement sequence it is optional to use the statement if the statement is not specified the next case is automatically executed now let us try to execute a sample program to understand switch case in a much better way you can see that this particular program is based on switch statement the switch condition is instrument now here the condition is that we supposed to select the instrument number four which happens to be flute now we have a various set of instruments present in our case which are piano trums guitar flute Uka volin and lastly the default statement which is invalid let us try to execute this program and see the output since we have provided our selection to be four which happens to be flute the output should be directly generated as flute now let us run this and see the output you can see that the program has been successfully generated and the selected instrument flute is been displayed in the console now moving ahead we shall understand the objectoriented style of programming in Java object-oriented programming is a programming style which is associated with programming Concepts such as inheritance polymorphism abstraction and encapsulation most of the popular programming languages like Java C++ Ruby Etc follow an object-oriented programming Paradigm as Java being the most sought after skill skill we will talk about objectoriented programming Concepts in Java an object based application in Java is based on declaring classes creating objects from them and interacting between these objects the first objectoriented programming style we will be discussing today is encapsulation encapsulation is a mechanism where you bind your data and code together as a single unit it also means to hide your data in order to make it safe from any modification what does this mean the best way to understand encapsulation is to look at the example of a medical capsule where the drug is always safe inside the capsule similarly through encapsulation the methods and variables of a class will be hidden safe to understand encapsulation in a much better way let us go through a simple example so this particular program is the example for encapsulation here we have encapsulate the data members which are the name of the user which is Ravi and account number of the user and email of the user and the am which is present in the particular users’s account now we will try to execute this program and print the details of the user you can see the program has been successfully executed and the account number of the user has been displayed here along with his name and mail ID along with the balance amount which is present in this particular account so this particular example was based on encapsulation now let us move ahead and understand the next type of objectoriented programming style which is inheritance as we can see in the image a child inherits the properties from his father similarly in Java there are two classes they are parent class which is also called as a super class or a base class and the next type of class is a child class which is also called as a subass or a derived class a class which inherits the properties is known as a child class whereas the class whose properties are inherited is known as the parent class inheritance in Java is further classified into four types they are single inheritance multi-level inheritance hierarchial inheritance and lastly the Hybrid inheritance the first one single inheritance in single inheritance one class inherits the properties of another it enables a derived class to inherit the properties and behavior from a single parent class this will in turn enable code usability as well as add new features to the existing code here Class A is your parent class and Class B is your child class which inherits the properties and behavior from the parent class followed by that we have multi-level inheritance when a child class is derived from a parent class which happens to be also derived from from another parent class that is a class having more than one parent class but at different levels such type of inheritance is known as multi-level inheritance if we talk about the flowchart Class B inherits the properties and behaviors of Class A and Class C inherits the properties of Class B so here Class A is the parent class of Class B as well as Class C so in this case Class C implic L inherits the properties and methods of Class A along with the class B this is what we call as a multi-level inheritance followed by this we have hierarchial inheritance when a class has more than one child class or subclasses in other words more than one child class have the same parent class then such type of inheritance is known as hierarchial if we talk about the flowchart then Class B and Class C are the child classes that are inherited from the same parent class that is Class A followed by that the last type of inheritance is called the Hybrid inheritance the Hybrid inheritance is a combination of multiple inheritance and multi-level inheritance since multiple inheritance is not supported in Java it leads to ambiguity this type of inheritance can be only achieved through interfaces we will execute examples for each one of the type of inheritances the first example that we will be executing is based on single inheritance so in this particular example we have the parent class as teacher and the child class as Hado teacher so here what we are trying to do is we are trying to inherit the properties of the parent class teacher into the child class Hado teacher let us try to execute this program and see the output you can see that the program is been successfully executed and the properties of the parent class have been successfully inherited into the child class that is the college name designation of the teacher and the main subject which he or she will be teaching now followed by this we shall try to execute the next type of example based on inheritance repeat so the next type of inheritance is the multi-level inheritance this particular example is based on multi-level inheritance where one child class inherits the properties of multiple parents at different levels one happens to be the first class two happens to be the next class and lastly three happens to be the last child class which extends the properties of one and two let us try to execute this program and see the output you can see that the program has been successfully executed and the output has been successfully generated on the console edure Rea happy learning followed by this we shall move ahead and understand the next type of inheritance so this particular example is based on hierarchial inheritance where two or more classes inherit the properties of one parent class so here the class one happens to be the parent class and class two and three are the child classes which are inheriting the properties from the same parent class which happens to be one let us try to execute this program and see the output you can see that the program has been successfully executed and the output has been generated which says Eda have be learning now we have discussed all the three inheritances which are single level inheritance multi-level inheritance and hierarchial inheritance the last type of inheritance which happens to be multiple inheritance or Hybrid inheritance happens to be not supported in Java to make this happen Java has come up with a new idea which is called as interface so we shall discuss about Java interfaces in the further chapters so the next Concept in object-oriented style of programming is the abstraction abstraction refers to the quality of dealing with ideas rather than events it basically deals with hiding the details and showing the essential things to the user if you look at the image here whenever you get a call we get an option either to pick it up or just to reject it but in reality there is a lot of code that runs in the background so we don’t know the internal processing of how a call is generated that’s the beauty of the abstraction therefore abstraction helps to reduce the code complexity you can achieve abstraction in two ways that is by using an abstraction class or an interface we shall understand abstract classes first an abstract class in Java contains abstract keyword what does the abstract keyword mean if a class is declared abstract it cannot be instantiated which means you cannot create an object of an abstract class also an abstract class can contain abstract as well as concrete methods now followed by that the next one is an interface interface in Java is known as a blueprint of a class or you can say it is a collection of abstract methods and static constraints in interface each method is public and Abstract but does not contain any Constructor along with abstraction interface also helps you to achieve multiple inheritance in Java we shall execute one single example for each one of these to understand abstraction in a much better way so the first example that we will be executing will be based on abstraction and the abstract class which is declared here is person so to declare this particular class we have used the keyword abstract so this particular abstract class consists of the following segments the first one is the name of the person gender of the person after this we have the class which inherits or extends this particular abstract class which is student so inside student we have a variable called as student ID which is declared as private now the elements from the abstract class will be extended into this particular student class and the output will be generated so let us execute this program and see the output you can see the program got successfully executed and the output is being generated here we have the name of the student as Priya and gender female similarly Karan Kumari John and their genders and also their degrees Karan Kumari and John are puring engineering and Priya is not studying so now let’s move ahead and understand interfaces in Java this particular example is based on interface and here the interface that we have declared is about a basic calculating functions which are add substract multiplication and divide so here we have declared the interface using the keyboard interface and as you can see we don’t have any Constructors inside this interface and followed by this this is our main class which is student which will implement the interface math to perform the addition subtraction multiplication and division operations let us try to execute this program and see the output now you can see the program got successfully executed and now the program is asking us to enter any two digits to perform addition let us enter 10 and 20 and you can see the sum has been successfully generated now it is asking values to enter to perform substraction let us enter 10 and five you can see the output as five now it is asking for values to perform multiplication let us enter 10 and 20 you can see the product as 200 and finally the division operation let’s enter 20 and five you can see the quotient is four so this is how face works now followed by this the fourth and last type of objectoriented programming style is none other than the polymorphism polymorphism means taking many forms where poly means many and morph means forms it is the ability of a variable function or an object to take on multiple forms in other words polymorphism allows you to Define one interface or method and have multiple implementations polymorphism in Java is of two types first runtime polymorphism and compile time polymorphism in Java runtime polymorphism refers to a process in which a call to an overridden method is resolved at run time rather than at compile time in this a reference variable is used to call an overridden method of a super class at runtime method overriding is an example of runtime polymorphism we understand method overwriting and method overloading in the further Concepts that is the advanced Java Jaa programming Concepts and followed by that we have the next compile time polymorphism in Java compile time polymorphism refers to a process in which a call to an overloaded method is resolved at compile time rather than at run time method overloading is an example for compile time polymorphism method overloading is a feature that allows a class to have two or more methods having the same name but with different arguments passed to the methods are different and like method overriding arguments can differ in number of parameters passed to the method data types of the parameters sequence of data types when passed to a method with this we come to an end of objectoriented style of programming in Java now let us continue with Advanced Java Concepts the first amongst the advanced concepts in Java is exception handling so first of all what is an exception an exception is an unwanted or unexpected event which occurs during the execution of a program at runtime that disrupts the normal flow of the program instructions error versus exception error an error indicates serious problem that a reasonable application should not try to catch whereas exception indicates conditions that are reasonable to an application that might try or to catch now how does a jvm handle exception there is a method called default exception handling whenever inside a method meod if an exception is occurred the method creates an object known as exception object and hands it off to the runtime system that is jvm the exception objects contain name and description of the exception and the current state of the program where the exception has occurred so this is how the exception is handled let us try to execute a simple program based on exception handling to understand it in a much better way so this particular example is based on exception handling in in Java so this particular program is related to divide by0 exception where the given number will be divided by 0 so anything cannot be divided by 0 so this particular exception is called as divide by0 exception and we cannot divide any number by zero let us run this program and see the output you can see that the program has been successfully executed and the output is been generated which says can’t divide it by zero now moving ahead the next Advanced Java concept is multi-threading multi-threading in Java is a feature that allows concurrent execution of two or more parts of a program for maximum utilization of CPU each part of such a program is called as thread threads are lightweight processes within a same process now let us try to execute a sample program to understand multi-threading in a better way now this particular example is based on multi-threading in Java we’ll be creating multiple threads like one or two and we will be seeing if that particular thread is existing and running or not now let’s try to execute this program and see the output don’t worry about the codes we will send you at your request if you provide your mail ID now you can see threads are created here thread one and thread 2 have got created and thread one is running thread 2 is getting started and also thread 2 is also running now you can see the functionalities of thread 1 and thre to and they are existing safely now with this let us move ahead into the next concept which is call by value and call by reference in Java call by value and call by reference in Java is just function calling firstly call by value means calling a method or a function with a parameter as value through this method the argument value is passed to the parameter while on the other hand call by reference means calling a method with a parameter as a reference which is the address of the value let us execute one example each to understand them in a better way so this particular example is based on call by Value method here we will be providing the values of A and B and call the function swap so this particular function is based on swapping the values between A and B so the values of a is 30 and the value of B is 45 so after calling this swap function the values of A and B will be swap now let let’s execute this program and see the output as you can see the programs has been successfully executed and before swapping the values were 30 and 45 now the values are exchanged now with this let us move ahead and execute an example based on call by reference so this particular example is based on call by reference we are performing the same swap function here but here we are providing the address of the values now let’s run this program and see the output you can see the program has been EX executed and the values have been successfully swapped now let us move ahead and understand the next Advanced programming Concept in Java that as method overloading and Method overriding method overloading is a feature that allows a class to have more than one method having the same name if their arguments list are different it is similar to Constructor overloading in Java that allows a case to have more than one Constructor having different arguments list now let us execute a sample program to understand method overloading in Java so this particular example is based on method overloading in Java here we have two different methods on the same name the name is ADD and the first method is having only two variables and the second method is having three variables now let us try to call these functions and see the output you can see both the functions have been executed and the sum is been displayed in the output console here so this was method overloading now let us move ahead and understand method overriding in Java in any object-oriented programming language overriding is a feature that allows a subass or a child class to provide a specific implementation of a method that is already provided by one of its super classes or parent classes method over adding is one of the major way in which Java can achieve runtime polymorphism now this particular example is based on method overriding here we have two different functions so the first function is in the parent class which is move and the second function is with the same name which is move that is existing in the child class that is dog both these methods are implemented in the main class and the functionalities of both the methods are included in the main class now let’s try to execute this program and see the output as you can see the program has been successfully executed and the output has been generated now let us dive into the conditional statements which are available in Java the conditional statements describes the compiler to carry out the execution of a particular code segment in a program until the condition is satisfied the types of conditional statements available in Java are ternary operator F operator LF operator LF ladder nested if breake and switch let us execute some examples related to all these conditional operators one by one firstly we shall execute an example based on if loop as you can see this particular example is based on if Loop here I’m providing a number which is a negative integer number minus 10 and here I’m providing an if condition which says that if the number is greater than zero then I should be printing a prin of statement which is number is positive else my interpreter will be printing the message the number is negative now let us try to execute this program and see how does it work as you can see the output has been generated which says the number is negative since the number which I provided was a negative integer number the condition here is failed and the message which was lying outside the if Loop which is the number is negative is been printed here now let us execute the next control statement where which is the lsf statement as you can see this particular example is based on lsf Loop here I’m trying to provide an integer value to the variable a according to my condition if the number which I have included results in a modulus is equals to 0 then I should be printing the message as even number else I should be providing the number as odd number now let us try to execute this program and see how does it work now as you can see the output screen is asking me to enter an integer value let us try to input an even number as you can see I have inputed an even number and I have received an output as an even number as the condition of the if Loop is been satisfied now let us try to input an odd number and see if the else Loop is working properly or not as you can see it is asking for an input now let us try to input an odd number which is 7 and see the output as you can see the output is generated as an odd number which means the L Loop is working fine now let us try to execute another example based on lcf Loop here I’m trying to execute an example based on ear I’ll be trying to input an year which the program will find out if it is a leap year or not the condition I have provided for if statement is if the ear modulus is equals to equals to Z and year modulus 100 is not equals to 0 or E modulus 400 is equals to equals to 0 then I must print the the message the ear you have entered is a leapia this means all the conditions which are stated in the if Loop must be true else it should print a message which says the ear you have entered is not a leap year now let us try to execute this program and see how does it work as you can see the output screen is asking me to enter an ear now let us try to enter a leapia as you can see the conditions which are provided in the if statement are true and we are printing the message the ear you have entered is leapia now let us try to provide an input which is not a leapia as you can see now it is asking for an Year let us input the present year which is 2019 and see if it is a leap year or not as you can see the conditions which were provided in the if Loop are not satisfied and the program is printing the message which is the year you have entered is not a leap year now let us try to execute some programs based on F’s ladder in this particular example we are trying to execute LF fladder condition as you can see here I’m trying to provide an integer typee input to the variable Choice according to the choices I have provided the first choice will select a sedan type class car and the second choice will select SUV type Class Car the third choice will provide Sports type class car and the fourth choice will provide crossbreed type class car if the provided input is neither of all these four then it will provide the last message which is invalid Choice as you can see the first F ladder will compare the choice if the choice is equals to equals to one then it will print the message sidan class and the next lsf ladder will compare the choice if it is equal to equal to two if it is equals to two then SUV class will be printed and in the next lsf ladder if the choice is equals to three then Sports class message will be printed and followed by that lsf ladder you’ll have the next ladder which will compare the choice to be equals to equals to 4 if it is true then crossb segment message will be printed if neither of all these conditions are true then finally an else block will print a final message which says it is an invalid choice now let us try to execute this program and see how does it work as you can see the program has been successfully executed and it is asking for a choice now let us try to select SUV type vehicle for that we need to provide an input as two as you can see the SUV class me is successfully selected now followed by this program let us execute nested F condition this particular example is based on nested if condition in this particular example I’m going to provide inputs for three numbers ns1 is = to 20 N2 is = to 30 and n 3 is equal to 10 and finally we have another integer type variable which is greatest using the netive condition we’ll try to find out the greatest number out of all these three as you can see the first if statement has the condition if N1 is greater than equal to N2 then it should enter an inner F Loop inside the inner F Loop we have a condition which says if N1 is greater than or equal to N3 then it should enter the loop and print greatest is equal to N1 else it should print greatest is equal to N3 after the completion of the if condition if neither of the statements which are provided in the if condition are true then the control will flow into the else condition inside the else condition we have an if condition which states if N2 is greater than or equal to N3 then it should print N2 else it should print n three now the final output will be generated along with the message which the largest number is and the greatest value now let us try to execute this program and see how does it work as you can see the output has been successfully generated and it says the largest number is 30 now let us try to execute an example based on ternary operator this part particular example is based on ternary operator as you can see I’m trying to provide an input to two numbers which are A and B which will be integer values this particular statement is our ternary operation this states that if the provided input a is greater than b then a should be declared as the greater number else if the condition is false then B should be declared as the greater number now let us try to execute this program and see how does it work as you can see the program has been successfully executed and it is asking for a value for the variable a let us enter 10 now it is asking for a value to the variable P let us enter five now let us see the output as you can see the output is 10 this states that the number a is greater than number B now let us try to execute an example based on switch conditional statements so this particular example is based on switch conditional statement here I’ll be ask a question as enter any month so I’ll be entering an integer value to the variable month if the input value is one then the program will execute January if the input value is two the program will execute February similarly if the input value is 12 then it will execute December and if neither of the cases match to the input which I provide then it will execute a default message which will be invalid input now let us try to execute this program and see how does it work as you can see the program has been successfully executed and it is asking me to enter any month let us try to input three as you can see see March is the message which has been printed by the program now let us try to execute an example based on break statement as you can see in this particular example I have provided an integer array which has the values 10 20 30 40 and 50 and here I’m trying to print the same array using a print statement with a provided condition if the number is equal = to equals to 40 then the follow Lo should terminate at that particular instance now let us try to execute this program and see how does it work as you can see the output has been successfully generated and the values 10 20 and 30 are printed but 40 and 50 are not printed because at this particular if statement we have provided a condition that if the number is equals to equals to 40 then the control flow should break so these were the few examples for conditional statements now let us continue with our Loops in Java we have three types of Loops firstly the for Loop next is wi Loop and finally we have do wi Loop the first type of loop is the for Loop the for Loop is a conditional flow statement which allows you to execute a specific code segment for a finite number number of iterations for a follow Loop we have three arguments namely initialization variable the condition variable and the counter variable which might be either increment or decrement now let us try to execute a program to understand for Loop in a better way before we execute an example of for Loop let us see the flow diagram of the for Loop here we have the initialization Step at the first and later the control flow will enter a conditional statement if the conditional statement happens to be true then a particular code segment or a particular set of statements are executed and after that the control flow enters into the increment or decrement variable and the variable will be incremented or decremented according to the condition provided once after the increment or decrement process is finished then the control flow will again enter into the conditional statement and verifies the condition if the condition is true then the same Loop will be executed else the control will exit from the condition statement now let us try to execute a program to understand this in a better way this particular example is based on fur loop as you can see I have a variable by the name ARR data which is of string type and this particular variable holds five elements which are John Jerry Ralph Jim and Tom which happen to be the names of the employees of a particular company now we will be using a follow Loop to display the names of these employees this happens to be the for Loop where we have the initialization variable which is I is equals to 0 and followed by that we have a conditional statement which is I is less than ARR data. length and finally we have an increment variable which is I ++ now let us try to execute this for Loop and see how does it work as you can see the loop has been successfully executed and we have our output here let us move on to the next example of fall Loop which is either called as enhanced for Loop or Advanced for Loop in this particular for Loop we will be having a conditional statement at the very beginning of the program the control flow enters the conditional statement and finds if there are any elements present in the array or not if the statement is true then the control flow will enter the initialization statement where a variable will be initialized once after the initialization is finished the control flow will directly enter into the condition statement of the program if there are any elements present in the program and the condition is true then it will execute the set of statements and initialize the local variable and once after the local variable is initialized then the control flow will continue into the same loop again if there are no elements present in the array then the control flow will exit the condition and stop and here as you can see if we come back into the initial conditional statement if there were no elements present in the array then the condition will become false and the control flow will directly exit the condition and comes to a stop now let us try to execute a program to understand this Advanced fall Loop or enhanced fall Loop in a much better way as you can see I have considered the same example of the list of employee names and here we are using the advanced for Loop which states that the control flow must enter the array and print all the elements present in the array according to their index now let us try to execute this program and see how does it work once after the control flow will enter into the array all the elements present in the array are printed until the last index value as you can see the program has been successfully executed and the output has been generated with this let us try to execute nested follow Loop before executing an example let us try to understand the flow diagram of the nested for Loop in a nested for Loop the program will begin at start and the control flow will encounter the first Loop statement after that the control flow will enter into the condition if the condition is true then it will enter into the inner fall Loop if the condition is false then the control flow will directly exit the first outer fall Loop now let us see if the condition is true once the condition is true then the control flow will enter into the inner for Loop and inside the inner for Loop it will encounter a condition statement if the condition is true then the control flow will start to execute a set of statements which are present inside the inner fall Loop and once after the statements are executed it will enter into the increment or decrement statements once after the increment or decrement is done the control flow will again enter into the condition statement and checks if the condition is true or not if the condition is true the same Loop will execute again and if the condition is is false then it will exit the condition statement and it will enter into the increment or decrement variable of the outer fall Loop once after increment or decrement of the out of for Loop is done it will again check the condition in the out of for Loop and if it is false then it will exit now let us try to execute a program to understand this in a much better way this particular example is based on nested for loop as you can see I have considered an integer type variable which is array which which is going to be a twood dimensional array and this particular Loop is our outter for Loop and this particular Loop is our inner for Loop once after the outer for Loop is triggered it will start the inner for Loop once after the execution of inner for Loop is finished then the inner follow will trigger the outer follow Loop for increment or decrement once after the increment or decrement is done then the condition is verified and the control flow will enter into the inall loop again once after the inall loop is executed then it will again trigger the aut ofall Loop and the condition is checked if the condition is true the process will continue else the control flow will exit both the fall loops and execute the statements which are present outside the loops now let us try to execute this program and see how does it work as you can see the program has been successfully executed and the data present in the row one and row two have been successfully printed here as you can see this is the first row which is row zero and this is the second row which is Row one and the data in the row zero is 1 2 and the data in the row one is three and four now let us learn about while loop while loop is a control flow statement that executes itself repeatedly until a given Boolean condition is satisfied while look can be considered as a repeating if statement let us see the flowchart of the vile Loop to understand this in a much better way as you can see in this particular example the control flow will begin at a decision if the decision is true then a particular segment of code is executed and again the control flow will enter into the decision box if the condition is true then the loop is repeated else the condition will exit the decision box let us try to execute an example to understand the V Loop this particular example is based on while loop here I’ll be providing an input to the integer type variable which is num and according to the while condition provided here we’ll find out if the input number is a prime number or not a prime number for that we have provided a condition in the while loop which states that if the variable I is less than or equal to the number divided by two then it should enter while loop and inside the while loop we have again a condition which states that if the number modulus I is equals to equals to Z then flag will be set to true and the control flow will be broken here and I will be incremented and outside the V Loop we have an if statement which states if the flag is not true then we should print it is a prime number else we should print it is not a prime number now let us try to execute this program and see how does it work as you can see the program has been successfully executed and it is asking for an input now let us try to input a prime number as you can see the output is 3 is a prime number similarly now let us understand do while loop two while loop is considered to be a conditional statement which is completely similar to the normal while loop the only difference is that the doy Loop has a Boolean or a conditional statement present at the end of the loop this makes the doy Loop execute at least for once to understand this in a better way let us check out a flow diag R in this particular example the control flow will first encounter the statement so this particular statement will be executed at least for once without checking the condition once after the statements are executed the control flow will enter into the conditional statement now the condition is checked if the condition is true then the same statements will be executed once more once after the statements are executed then the condition is checked again if the condition is true then the same Loop will be continued else the control flow will exit the condition now let us try to execute a program to understand this in a much better way this particular example is based on doile Loop so in here I’ve considered an array into which I’ll be entering elements the process of entering the elements will continue until I provide an input zero and finally all the numbers which I have provided into the AR AR will be summed up and the final summation of all the elements present in that particular array will be printed now let us try to execute this program and see how does it work as you can see the program has been successfully executed and it is asking me to enter the numbers let us try to enter a number other than zero my first number will be 9 and after that let me enter another number other than zero which will be 7 followed by that 5 as you can see it is continuing to accept the numbers which are other than zero and now let us try to enter zero and see what happens as you can see the loop is been terminated and the sumission of all the numbers which are provided into the array are summed up and the summation is provided as 25 so with this let us execute the next type of loop which is infinite Loop before we continue into the infinite Loop let me provide you a simple example to understand while loop and do V Loop in a much better way you must be familiar with the Road Runner Cartoons as you can see the Road Runner here is running on V Loop and the coyote which always tries to hunt the Road Runner is trying to execute itself using a do while loop so here the condition was checked first and the Road Runner is safe and in the case of coyote it was using a dowi loop and executed itself without checking the condition and it is outside the CLI so this is the basic difference between the V Loop and the two vile Loop now with this let us enter into the infinite Loop the infinite Loop is not actually a practically designed Loop instead it is a situation where a condition of the loop fails and the execution process until you stop it manually let us try to execute a program to understand infinite Loop in a much better way so this particular example is based on infinite Loop where I have provided a v Loop so here the condition of the V Loop is always true and the control flow will enter into the V Loop and will print the message edura let us try to execute this program and see how does it work as you can see the program has been successfully executed and it is continuing to print the Eda message continuously now I have to stop this manually and exit the infinite loop as you can see the infinite Loop has been successfully terminated now objectoriented programming so guys this is one of the software design methodologies that how we going to write a software solution in the industry and it’s followed widely so what is this objectoriented programming it’s a methodology or a paradigm using which we can design our software Solutions now there are two major Words which you need to focus on the first one is object and the second one is class now why object comes first in class later because of the term itself object oriented not class oriented so we’ll discuss on these objects and classes so guys object is a real world entity when I say a real world entity what does it mean it means anything which you can see touch feel is an object for example la laptop mobile phone chair table fan anything even you and me are the objects so every object will have some State and behavior what’s a class class is a blueprint or I can say it’s a drawing of an object so how an object will look like how object is going to be represented that we can do by having this guy called class so let’s say there is a class called Dog which is a represent ation how a dog will look like so we will have a representation in two of the ways the first one is properties and the second one is Behavior now properties are also referred to as attributes for example what is the weight what is the age what is the size color breed so these are the properties I can even say that they are the attributes which are linked to your object Behavior goes like eat sleep run back Etc and Etc so let’s say a nose over here is used for having a process called breathing or respiration respiration is a behavior so we have object with the properties and behavior so object will have properties object will have behavior and these properties and behavior will be written in the class to describe the object so having object oriented programming approach we can have different type of objects so this properties will remain same but the value for their properties will change now the size age color and breed for all the different dog objects it’s going to change guys right so even when I talk about something like employer to a company let’s say plumber electrician Carpenter let’s say HR let’s say software engineer now the different employees they have the same details that’s like what is their employee ID what is their name what is their age what is their fold but the data for these properties will be different now same way the sleep pattern would be different for different people right so this is what we are going to understand that different objects they will have the properties same properties but with the different values right so I can say an object is a container which is containing properties and behavior both now how we are different from procedural approach the way we used to write programs in C so how is it different right let’s have some comparisons coming in here so object oriented programming is a bottom to up approach what does this mean this means you think of an object first and then you start coding you think of data first top down approach is like where you just start writing the program we modularize our program into objects whereas in procedural approach we need to write functions for security we have access modifiers we got default private public protected but there we have no excess modifiers objects can move and communicate with each other through member functions but whereas in functions we don’t have objects so data is just moving freely so having access modifiers are a software solution can be designed with much more security right as compared to procedural approach and lastly we have have features like overloading overriding inheritance and many other features which objectoriented programming will give us so that we can very easily design our softwares which procedur programming will not do that now there are four major fundamentals when it comes to object oriented programming structure guys we got encapsulation abstraction polymorphism and inheritance so these are the four major building blocks of any object oriented programming language right so we even call them the pillars for the oops so we’ll discuss them one by one to begin with what is inheritance so guys inheritance is where the property of an object will be acquired by the other object we have a relationship called is a relationship so it’s a parent child relationship so having this relationship let’s say animal is the parent mammals reptiles amphibians and birds they are the children so this is is a relationship bird reptile mammal is an animal right that is how we are relating them we also call this concept as generalization because here we are generalizing so many different types as one single type right so we call animal as super or the parent class whereas mammals reptiles amphibians Birds they are known as sub or the child classes so guys let us have one example on inheritance and therea let’s try to code it so consider that there is an object structure called Product so product is going to have a product ID product will have a name product will have price so these are few of the attributes associated with the product right now I’m taking this use case let’s say we are to develop an e-commerce solution for an e-commerce solution now there will be a product which can be an LED TV so LED TV will extend the product right so is a benefit of extending LED TV from product so you’ll get all these attributes in the LED TV as well right so we just need to write additional parameters so we will come here and say what is the BR and what can be the technology for LED TV and any other attributes if you want to have we can have a mobile which can extend the product so we can have Ram what is the OS what is the SD card size and many more attributes so extension is what is called inheritance so if we don’t have extension so how the things will work let’s see that with no extension we need to rewrite a similar code snippet again and again so you see p name price brand technology is the data associated with LED TV same is with the mobile so P name price they are the common attributes so when we know that we have some common attributes or common properties which will be Reus again and again this is like repeating the code again and again whereas when we are coming up with this structure here so we are reusing the similar code again and again you have product as one guy and you now keep on reusing it again and again so let’s try to come up and write one small program here and understand this example so I’m going to say a new Java project so we are using Eclipse as an ID so you can use intell J developer so whatever you want to use you can use that let’s say a new Java project I’m going to say oops so in this objectoriented programming structure on the SRC I’m going to do a right click and say a new class now this new class over here let us name it inheritance app with the main method so I’m going to write a package let’s say cod. edura so this package name cod. Eda contains inheritance app with the main method right so we know what is main main is executed by jvm right when my program will run now let’s come here and write this class as product and I’m going to write one Constructor here here with says ceso product object constructed and thereafter let’s have some attributes we got product ID what is the name and let’s say what is the price so we can even have more attributes but I got these many details so I’m going to write methods now so this is Constructor the default one so the methods I’ll say void let’s say set product details where I will say let’s take three inputs P ID name and price so I’m going to say this. p ID is p ID this dot name is name and this dot price is price so we call this guy as a method which is setting the details into the the product object I can even say to write data in product object we have this method we also call the methods as behavior and attributes are also referred to as state same way I’ll have show product details where I will come and say cisu let’s see plus product just give me a moment all right so I’m going to say product ID this and thereafter I’ll just have this coming in now I will say ciso name back SLT plus the name the next way we have price back SLT and the price so this is the method which will read the data right so to read data from product object so we call this structure this class product is textual representation how an object will look like in the memory right you need to understand that this P name price they do not belong to the class they belong to object so whatever we write in class is in actual property of object as we are describing the object right so using class you are textually describing the object so P name price they do not belong to class Constructor the methods they belong to the object so if you want something to be property of class if you want anything which should belong to class make it static right so this is what you need to do now coming here guys let us create the product object so create an object what object I want product object let’s say product is a new product so remember this statement is an object construction statement but product is not an object it’s a reference variable which holds the hash code of the object in hexa decimal notation so what does this mean this means that if I will say product is plus product so when you run this code here as Java application you will get to have a product object constructed and this guy reference variable is giving you this 7852 e922 so we get 7852 e922 as a hash code now this might vary from system to system right so we say product is a reference variable right so product is a reference variable not an object it is pointing to the object which is created in the memory in the heat so next is writing data in object so I will say product dot set the product details let’s say product ID is 101 let’s say name is iPhone x let’s say the price is 70,000 so this is the data within the object which we have wrote and now let’s say reading data from object you will say product dot show the product details when you run this code here let me comment out this structure here and here after writing the data I’m just going to put up this C statement data written in product object so just to show this so we got product object constructed and then data written in product object and this is data with respect to your product object so this is how we get to see data within the object write and read operations very basic structure of an object now we can even write the data directly coming here let’s write the data directly so I can say product dot P ID let me create another object so I’m going to say a product this time let’s say this as product one is a new product or let’s say this is a product two this guy is product one and here I I’ll say Product 2 do p is 2011 Product 2 dot name is Nike shoes Product 2 dot price is let’s say 5,000 so this is writing the data directly sorry my bad so this is integer so you’ll say Product 2 dot show the product details so just going to do an empty print line here for the correct output so what you see is product object constructed we are not writing the data using any method we are writing the data directly and you see the data read operation happening again so if you don’t want the data to be written directly you can make the attributes go as private so having your attributes as private so you are limiting the user not to write the data directly you see you are now getting an error here so I’m not going to make everything private let’s say the product ID is private so when you have any attribute marked as private you will not be able to access it so this is an error now since attribute marked as private cannot be accessed you cannot access an attribute which is marked as private so what we will do now so we can have an indirect way to access P ID we can create a method which says set the P ID you take integer as input and you say this dot p is p ID so guys when I’m using this so This Means reference to the current object so this means reference to current object so the left hand side P belongs to object right hand side P belongs to the method set P ID okay so LHS belongs to the object and rhs belongs to Method so we can similarly say in get P which returns back the PID So when you say set p and get P ID what does it mean so these are the special methods we call them Setters and getters seter and getter is required when you have your attribute marked as private so you have some indirect way of capturing these details in your object so now I can come here and say Product 2 do set the P ID as 2011 now this is going to work for us so I think the basic product object structure is clear to everyone now moving ahead into our use case where we are going to extend the product right so let us take one example of mobile and extend the product now I’m going to say a class called mobile with it extends product so this is known as a relation where we say mobile is a product and here mobile is child and product is parent so what I’ll do is I’ll create a mobile Constructor and say a ciso mobile object constructed right so this is how we going to deal with the object now let us comment out this code snippet in the main so instead of having the direct product object I’m going to write a mobile object now so I’ll say a mobile is a new mobile now whenever I am requesting for the mobile object right so we are requesting to get mobile object constructed so this is what we are doing here now when you write this instruction what you see is before your mobile object is constructed project product object is getting constructed so it’s like parent object is constructed before the object of child right so what is the behavior here so product object gets constructed before the mobile object so as a rule of inheritance it is like firstly you get the parent object constructed and then the mobile object constructed so guys I hope this is clear right so what is happening here we have parent object constructed before the mobile object as this becomes the rule to inheritance and we say this is object to object now you will be glad to see that everything whatever product had will be acquired in Mobile right so we can come here and we can say mobile do set the product details mention the P ID let’s say 301 mention the name let’s say iPhone x mention the price let’s say 70,000 and then you say mobile dot show the product details when you run this code you get to see that a product object constructed mobile object constructed data written in the product object and you see the details flowing for you so guys I hope this is clear right so whatever you had in the parent you got in the child right so we have the things coming up for the child and we can very easily access those things now this product over here we can have something like let’s say string what is the OS int what is the RAM and int what is the size of SD card right so let’s say SD card size now these are additional attributes additional attributes of mobile other than the product so these are the additional attributes now what I’m going to do is I’m going to have this guy over here called set product details let’s do a copy here and let’s try to do a paste here now I’m going to say string OS int RAM and int SD card size so here I’ll say this dot OS is OS this dot Ram is RAM and lastly this dot SD card size is SD card size you see p is not accessible we getting an error here it says it’s not visible because it’s marked as private so private is not inherited and visible into the child all right I’m going to make this p as default so we talk about the excess modifiers later

    but we are focused on understanding the oops here now we actually have this same method name set product details right so we have redefined the same method from the parent into the child with different inputs so we got two methods in the child so we have now two methods in the child one from parent and one of child right so both are different as in based on inputs right even though name is same even though name is same they are different based on the inputs so whatsoever belongs to the parent that is inherited into the child and now in the a child we have the same method coming up again why we did this because we wanted to set the data for the additional attributes so this is known as method overloading same method name with different inputs so what I can do now is why to access the parents method here I can have my own method mobile do set the product detail with the p ID let’s say 301 name let’s say iPhone X price let’s say 70,000 OS let’s say iOS Ram let’s say 4 SD card size let’s say 128 and thereafter I’m going to say mobile. show the product details now when you run this code guys what we see is it says data written in product object so what I can do is I can just manipulate this and I can say data written in Mobile object when you run this code you see data written in the mobile object but whatever you are reading is the one which was inherited from the parent so this read method show product details is inherited from the parent and that is being used here so hence we cannot see more than three details I’ll redefine this method as well so let’s redefine show product details as well but here we have same inputs two methods one from parent one in child and we have same signatures when I say same signatures it means they have same input details so what I’ll do is I’ll extend the data here so we’ll have OS we’ll have RAM and we’ll have SD card so we’ll have this OS RAM and SD card science now what you will observe is that when the show product details will be executed so we got two show product details method one in the parent one in the child so child method will be executed and not the parent method so this fundamental over here is referred to as overriding very very beautiful concept guys so overriding versus overloading method overloading whereas this is Method overriding so same method name with same inputs in parent child relationship so only in a parent child relationship now when you run this code here what you see is the other details also coming in now we know the reason that why we need to overload a method why we need to override a method we need to overload a method so that we can write the exact data which we want so we are customizing this method in the child we are overwriting the methods so that we can display more data so we are customizing the methods in the child right so overloading and overriding is what where we are having customizations to predefined methods we don’t want these predefined method we want the way we want to present the data and we have it in front of us so this is a very quick introduction to objectoriented programming structure guys so now let us come here and understand the next part so inheritance is where one object will acquire the properties of other object it’s is relationship the parent child relationship so synthetically you need to come up and say extends so which is extension right what are the advantages code reusability so we are reusing the code having an extensibility we are using overriding we are even having the features to hide the data if you privatize any attribute you won’t be able to inherit it so we got different types of inheritance so you got single level multi-level and hierarchical where single level is one parent one child multi- level is parent to child to grandchild and hierarchy is having one parent with more than one children multiple inheritance that’s opposite of hierarchy is not supported in Java so there is no virtual keyword there is no pointer technique in Java by which we can solve the problems like dreaded Diamond here so in single level inheritance we got one parent and we got one child so arrow is upwards because B is an a right so it’s extension relationship then we have multi level parent to child child to grandchild and thereafter guys we got hierarchy where one parent with the multiple children so remember we don’t have support for multiple inheritance in Java that’s like the reverse of this hierarchy multiple parents having the same object next is polymorphism so guys it is having the same name with the different different definitions so you can say mothers they are the best example of polymorphism they handle kids they handle office they handle home more than one form right so here polymorphism can happen in two different ways compile time and runtime so what is compile time versus runtime so compile time means overloading we call it static polymorphism so compiler will ensure which method will be executed for which function call so there are some rules to overload the method name should be same and the argument list or the inputs to the method that should be different and unique so the other form is runtime polymorphism that’s like Dynamic polymorphism where we got Method overriding All right so overriding we got some rules coming in here so the rules to override is where the method name should be same input should be same return type is something which must be same or the subtype of the overden method and excess level it must be same or more restrictive now what is abstraction guys so abstraction is achieved using abstract classes and interfaces so this is where we achieve a runtime polymorphism in a different approach right so here we have an abstract class so an abstract class is created using an abstract keyword so here we will have the methods with abstract keyword we even have non-abstract methods this is a class for which you cannot create objects you can have Constructors you can have static methods but we cannot create the object right so you can even have the final methods substract class X as a template for the various methods and definitions to be executed so moving ahead so the better version of abstract class is an interface so interface is a blueprint for a class which will contain some constants and Abstract methods so it will enable a multiple inheritance and helps in achieving the loose coupling so a class can extend a Class A Class implements an interface now this is one of the major differences between classes and interfaces so we got interface to interface inheritance as well so guys abstract class and interfaces so as in practically we’ll see it in our other sessions so lastly what is encapsulation where we are going to privatize the data so we can have an attribute we can mark it as private and when the attribute is marked as private it cannot be accessed so we need Setters and Getters so as to access this private data so so encapsulation leads to data hiding so it is focusing on security what are abstract classes in Java abstract class in Java act as a boundary between the implementation method and its functionality it is used to exchange the functionality between the concrete class members and the abstract class abstract classes are considered as those classes that hide the method implementation details from the user and show only the method functionality they declare using the keyword abstract and these methods can include abstract as well as non-abstract methods in them with this let us continue with the next topic which is why do we need an abstract class in Java we need abstract classes in Java for the following reasons firstly default functionality abstract classes are designed to provide a default functionality of the defined method for all the sub classes next template abstract classes provide a template for future specific classes next code reusability abstract classes allow code reusability followed by that we have separate method definition abstract classes separate the method definition from the inherited sub classes next we have loose coupling abstract classes help the users to achieve loose coupling between the class members and class methods last but not the least which is dynamic method resolution abstract classes support Dynamic method resolution in runtime with this let us jump into our next topic which is rules for using abstract classes in Java to use the abstract classes in Java we are supposed to follow the rules defined as below firstly in any project or a program always remember that the abstract class cannot be instantiated the next one is an abstract class can also accommodate final methods next an abstract class must always be declared using the abstract keyword followed by that an abstract class can have both abstract as well as non-abstract methods finally it can have Constructors as well as static methods so these are the rules required to use abstract classes in Java with this let us now understand the ways to achieve abstraction in Java abstraction is one of the objectoriented programming practices to achieve abstraction in Java we have two methods first using an abstract class and the next method using an interface now let us understand the Syntax for using abstract classes in Java for defining abstract classes and Abstract methods in Java we have the following syntax firstly let us understand the syntax to declare an abstract class in Java for example here I’m declaring a class by the name edureka and I’m using abstract keyword here here now similarly let us understand how to declare a method using the syntax the Syntax for declaring a method is almost similar to the syntax of declaring a class the only change is that the use of parenthesis and a semicolon with this let us try to execute some practical examples of abstract classes in Java to understand the terminology in a much better way the first example would be student information after this we shall execute two more examples the examples will be same but implementation will be different first we’ll use an interface to execute the same program followed by that we’ll execute the same program using an abstract class this is done to understand the difference between an abstract class and an interface now let us execute our first example as you can see this particular example is based on abstract classes you can see that I have used the abstract keyword to declare this particular class which is person this person class includes private data members which are name and gender similarly I have also included another method which is an abstract method and I have declare that method using the abstract keyword and inside the abstract method I have to written some data related to the person class now let us move on to the next class which is the student class this particular student class is a normal class and this student class is extending the abstract class which is person inside the student class we have some information related to the student which are student name student gender and student ID here we will be extending the abstract class which is person into the student class and from there we’ll be extending the details of name and gender now let us try to execute this program to understand this in a much better way as you can see the program has been successfully executed and the data is also been displayed here according to the output the names of the students are Priya Karan Kumari and John and their respective genders are also been displayed here now let us try to execute an example based on interface the program that we decided to execute based on both interface as well as abstract class is to find out the area of two shapes which are rectangle and circle now let us try to execute this program and see how does it work firstly we shall Implement interface as you can see in this particular example I’m using interface and inside the interface I have described two methods which I draw and area here we have provided input to both the methods which I draw and area for both the shapes which is circle and rectangle now let us try to execute this program and see the output as you can see according to the data provided to both Circle and rectangle here are the outputs of their area now let us execute the same program using an abstract class as you can see this particular class is the abstract class which is shape and inside the shape we have two methods which are the area of the shape and draw function followed by that we have the class rectangle which extends the abstract class which is shape and inside the rectangle class we are implementing draw method as well as area method similarly followed by the rectangle class we have Circle class which extends shape abstract class and inside the circle class we also have draw method and area method finally we have the class edura where we’ll be providing the data for rectangle class as well as Circle class now let us try to execute this program and see how does it work as you can see the program is successfully executed and the area of rectangle and circle are generated you can see that the area of rectangle is 600 followed by that the area of circle is 628 with this let us move on to our next topic which is the difference between an abstract class and interface the first difference between abstract class and interface is abstract class can include abstract as well as non-abstract methods while interface can have only abstract methods the next difference is abstract class includes non-final variables on the other hand interface can include only final variables followed by that abstract class can include static non-static final and non-final variables compared to abstract classes interfaces can include only static and final variables the next difference is abstract class can Implement an interface while interface cannot Implement an abstract class now we shall discuss how to implement abstract classes as well as interface to implement an abstract class we use the keyword extents and to implement an interface we use the keyword implements the next difference is abstract classes can extend Java classes and interfaces while interface can extend only an interface the last difference is that members can be private as well as protected in an abstract class while in an interface members are declared as public by default why do we need inheritance to understand this let us consider a simple example of two different watches first one is a classic dial watch and the next one is a modern digital watch the functionalities defined for a dial watch are display date display day and display time similarly let us get into the advanced version of the watch which would be our digital watch the function alties required for the digital watch are as follows display day display date display time light set alarm and set timer as you can see three functions from the dial watch are repeating here in real time this would be considered as code redundancy you might avoid by removing the extra code from the digital watch but it results the digital watch will be missing the three functionalities which are display time display date and display dat you might try to keep them in the digital watch and try to remove those from the dial watch but again in the real time scenario you will lose the existence of dial watch the only solution for this problem is by applying inheritance to both the classes where the dial watch will act as a parent class and digital watch will act as a child class the child class will inherit all the methods and properties of the parent class and successfully reduces the code redundancy improves the readability and the functionality of the code to an exponential level now that you have theoretically understood what the term inheritance means let us now get into Java inheritance and understand it through an example like any other programming languages Java inheritance is also known as a procedure where the child class inherits all the methods and properties of the parent class as you can see the tablet here is inheriting the properties both from laptop and a mobile phone this was a pictorial example let us see a practical example through a code in this particular example we have two classes the first one is teacher which acts as a parent class inside the teacher class we have two strings teacher and edura along with that we also have a print statement which has teaching message inside it now let us get into our child class Hadoop teacher happens to be our child class and in our child class we have a string called as spark now we will be performing a print operation with which has to access the college name designation and the main subject as you can see we have the main subject inside our child class and college name and designation of the lecturer in our parent class now we’ll try to inherit the parent class through the keyword extents and this is the name of our parent class which is teacher now let us try to run this program and see the output as you can see we have successful inherited both the messages from the parent class which is Eda which happens to be the college name and the designation which happens to be the teacher and the main subject which is spk this is been displayed from the child class and the message from the parent class which is teaching now that you have understood Java inheritance in a practical way let us now get into the types of inheritances supported by Java Java basically supports four types of inheritance namely single inheritance multi-level inheritance hierarchial inheritance and Hybrid inheritance let us discuss each one of them in detail firstly single inheritance single inheritance is a procedure where one child class inherits the methods and properties of one parent class in simpler words the whole system consists of only two classes one parent and one child let us understand this through an example here we shall consider two classes one class will be person and the other class will be employee where the person class acts as a parent class and employe class acts as a child class let us execute this program practically you can see here that this happens to be a parent class which is person which has two different functions the first function is set age and the second function is set name now let us move on to our child class inside our child class we have one method which happens to be set employe ID the methods set name and set age which are in the parent class are required in the child class as well so let us try to inherit those into this child class so this is the code for inheriting the properties of the parent class into the child class where I’ll be inheriting the properties of get name and get H into my child class to display the data of a particular employee in an organization now let us try to execute this program and see how does it work as you can see the properties of get name are in inited and I have successfully named my employee as ragir and I have set the age of ragir as 23 and accordingly his employee ID is EMP 11101 so this was single inheritance now let us move into the second one which happens to be the multi-level inheritance multi-level inheritance is a process where multiple child classes inherit the properties from their preceding parent classes to understand this in a better way let us consider a simple example as you can see in the picture Class A is the parent class and Class B and Class C happens to be the child class here child Class C is inheriting the properties from its preceding parent which happens to be Class B and Class B is inheriting the properties and methods from the superior class which happens to be Class A this was a pictorial representation now let us see a practical example through a code as you can see this is our program and calculator is a parent class similar to class A in a picture which performs basic mathematical operations such as multiplication subtraction addition and division now let us move into our class B which would be the class advanced advanced class can perform a little Advanced operations such as modulus of a number square root of a number and square of a given number and now let us move on to our child Class C which happens to be the area so this class can perform a little Advanced calculation such as finding an area of a geometric figure such as Square rectangle and circle now we need a calculator which could perform all these operations in one single class for this the only solution would be multi-level inheritance where we will be inheriting all the methods and properties of superior parent class into the succeeding child classes now we have tried to inherit all the properties and methods of the preceding parent classes into the last child class let us try to execute this program and see the output as you can see the values are already passed to all the functions now their respective outputs have been successfully displayed so this was multi-level inheritance now let us move on to the next one which happens to be the hierarchial inheritance hierarchial inheritance is a procedure where we’ll be having multiple child classes and all the child classes will be inheriting the properties from the major parent class here let us assume an example of a restaurant we shall have four different restaurants here the first one would be a parent restaurant and the other three will be Chinese restaurant Italian restaurant and Mexican restaurant the parent restaurant will be having various types of ingredients which the child classes might need in their dishes as well let us see this practically to understand this in a better way so this would be our parent restaurant which will be having the ingredients like bread vegetable stew and cheese now let us get into the first child class which happens to be the Chinese restaurant the Chinese restaurant already has noodles and Momos to cook their special dishes they might need vegetables too and cheese similarly let us get into Mexican inside Mexican they already have nachos and sweet potato for their special dish they might need cheese and vegetables toew or even bread for making some better dishes similarly let’s get into Italian which have pasta and sausage available so to prepare their special dishes related to sausage they might require bread and to prepare their special dishes related to pasta they might require cheese and vegetables too so how does all these three restaurants get their ingredients from simple they apply hierarchial inheritance and inherit all the properties from the parent class and cook their special dishes now let us run this program and see how does it work as you can see all the child classes have successfully inherited their parent and the data is also been displayed the first one is the Italian restaurant so they have requested for bread vegetable stew and cheese from the parent class and they say that they already have pasta and sausage available with them now the second one happens to be the Chinese restaurant and they say that they have requested for bread and vegetable stew from the parent class along with cheese and they have noodles and Momos already available with them similarly the final child class happens to be the Mexican restaurant and they see that they have also requested for bread vegetable stew and cheese from the barent restaurant and they already have nachos and sweet potato available with them so this was an example for hierarchal inheritance now let us move on to the next one which happens to be the Hybrid inheritance as simple as the name suggests Hybrid inheritance can be a combination of any other inheritances as you can see in the picture displayed here you can see multi-level inheritance here and hierarchial inheritance so these are the two combinations of two inheritances possible in Java The overal Inheritance pattern can be called as a Hybrid inheritance now you might be wondering about another possibility in inheritance which might be obviously the multiple inheritance here one single child class can inherit all the methods and properties from two or more different parent classes but sadly in Java this results in a diamond problem and could not support multiple inheritance it is possible to achieve this Behavior through Java interface with this we shall move to our next topic which is EA and has a relation is a relation basically deals with inheritance and interface any child class which inherits or implements a parent class is said to be in a easa relation coming into ha a relation it uses instance variables to reference the objects of the parent class this might be a little confusing let us understand it practically as you can see the BMW car is a vehicle which is inheriting the properties of a vehicle so it happens to be in a East a relationship with vehicle and coming into has a relation it has an engine so the BMW car is referencing to an object of engine class and said to be having an has a relationship with engine class let us execute a program and see this practically as you can see my child class is BMW and it is extending to the parent class which is car and also extending to another parent class which is Java so PMW class is in an EA relationship with car and in an has a relationship with engine class now let us execute this program and see how does it work as you can see the class BMW has successfully inherited both the classes which are car and engine and the properties which it inherited from car class is the color which is blue and the speed from the engine which is 256 with this let us get into our last topic which is Method overloading and Method overriding method overloading is defined as a procedure of having two or more methods with the same name within a single class provided the argument list must different method overloading is divided into three types the first type would be having different number of parameters and the second one will be having different types of data type and the third one will be the variation in the sequence of the parameters with different data types let us see each one of them practically this particular example is the first type of method overloading here in our parent class you can see we have two functions on the name of display so the first display method has one parameter which is a character type and in the second display method we have two parameters one is of a character type and the other one of an integer type so from the child class I’m inheriting these both methods and one of the object has one single parameter which happens to be the character type and the second object has two different parameters one of it is a character type and the other one is an integer type let us execute this program and see how does it work here you can see the output the first one is calling the first display method with one single parameter and the second one is calling the second display method which has two parameters now let us see the second type of O loading in here we have two display methods in a parent class one of the method has character type of data type and the other one is having integer type of data type in a child class I’m passing character type of data to the first display method and integer type of data to the second display method now let us try to execute this program and see how does it work as you can see the data has been successfully displayed the first one is calling the method display with character type and the second one is calling the method with integer data type now let us see the third type of method overloading in here we having the same two display methods in the parent class one of them is having the character data type in the first position and integer data type in the second position and vice versa in the second display method now similarly through my child class I’m passing two different parameters the first object will be passing character data type in the first position and integer in the second one and the second object in the vice versa now let us execute this program and see how does it work as you can see the message from the first display method is been executed here which shows that the first object is referring to the first display method and the second object to the second display method now let us take a step further and understand type promotion as well in one of my display methods I have provided character and double attributes whereas float and character attributes in the object The Interpreter will implicitly promote the data type of the object from float to double this type of promotion is valid since there is no other display function with matching attributes now let us try to execute this program as you can see the object has selected the first display method and upgraded the float to double the message from the first display method has been displayed here which happens to be function one now let us try an example where we should have a display function with matching attributes and verify if the promotion is accepted or not as you can see the object display is passing two data types which is an integer one and Float one and here we have the first display function with float and integer data types let us try to execute this program as you can see we have the output now in this scenario there is no type promotion since there is already the data type which is available and the parameters are matching with this we shall get into method overriding method overriding is a process where both the child and parent classes have the same methods there are also some rules for method overriding which say the argument list of the child class should match the parent class the second one is access modifier of the class should be less restrictive than the parent class and the third one is local parameters cannot be overridden local parameters in the sense the parameters with access modifiers as private static and similar ones let us see each one of these rules practically so let us execute our first example here we have defined two objects which are pointing to the display methods the first object implements the display method from the parent class whereas the second object implements the display method from the child class this happens in the runtime hence this process is called as runtime polymorphism let us execute this program and see how does it work as you can see the display methods are been implemented and the data present in the methods are also displayed now let us discuss about some key points of method overwriting first one the argument list of the child class should always match the argument list of the parent class second one the access modifiers of the child class should be less restrictive when compared to the parent class let us see this practically as you can see in my example the child class has an access modifier as public and the bearing class is having an access modifier as protect Ed this program will be executed without any errors let us try to execute this as you can see the data has been also displayed which says the child class method has been executed now let us try to change the access modifiers now the only change what I have done is I have changed the access modifier of the parent class to public and the access modifier of the child class to predict it as you can see the eclipse editor is throwing an error which says that the child class cannot have an access modifier which is highly restricted than the parent class now let us discuss our last topic which happens to be about the super keyword in method overriding the keyword super can be used in the child class to call the method defined in the parent class with its name the super method is used to invoke the Constructor defined in the parent class let us see this practically using super keyword I’m calling a method by name My Method from the parent class now let us try to execute this program and see if it can successfully call the method or not as you can see it has successfully called the method from the parent class and the data is also been displayed what is polymorphism to understand understand this phenomena let’s take an example of a chameleon I hope you all know what exactly is a chameleon in the real world you might have seen this creature changing its color based on its requirement to hide itself from the Predators if someone inquires you how does it do that you can simply say because of its polymorphic nature similarly in the programming World Java objects possess the same functionality where each object can take multiple forms this property is known as polymorphism in Java where poly means many and morph means one or more forms in this session for today we shall discuss one of the key concepts of object-oriented programming which is none other than polymorphism now that we have understood what exactly is polymorphism let us go through some examples here you can see the image the person in the middle can act in multiple forms firstly you can see the same person being a student followed by that he will become an employee of a company and also he would like to play some sports followed by that the same person is acting as an interviewer for a fresher then the same person is acting as a normal human being who is purchasing groceries and followed by that the last example of the same person being a family man you can see the same person is playing different roles at different situations this is called as a polymorphism of a human being now that we have finished the first example let us go through the second example of polymorphism here suppose you have to save contact numbers of different persons now it’s not possible that every single person in your contact list have two numbers but few of them may have so in this situation let us assume that you have a friend by name Jack and Jack has two numbers you have saved the first number and in case if you want to save the second number you don’t have to create the new method called create contact and save the second identity number you can just call the same create method what you have created for the first one and save the second number with the same name here you have saved Jack contact ID with the same name but you have two different numbers so this example is based on polymorphism now that we have discussed two examples of polymorphism let us move ahead to the next topic where we will discuss about the types of polymorphism available in Java so there are basically two types of polymorphism available in Java they are static polymorphism and dynamic polymorphism let us understand each of these in a bit detail firstly the static polymorphism a polymorphism that is resolved during compile time is known as static polymorphism method overloading is an example of compile time polymorphism so what exactly is Method overloading method overloading is a feature that allows a class to include two or more methods to have the same name but the different parameter list to understand this in a bit more detail let us go through a sample program so you can see this particular example is based on polymorphism or method overloading so in the class called my class we have created two methods the first method is my class which has the statement to print the message bricks followed by that in the same class we we have another method called my class which has the same name but inside this method we have a variable I of integer data type and inside the second my class method we have a message which says building new house that is plus I plus feet tall we are giving the height which is I so this height will be replaced here and the message will be printed so followed by this we have the void in this void method which is the info we have a message again which says houses height feet tall so this height will be replaced by the height we provide in the second my class method and followed by that we have another void info which prints the message saying S Plus houses height feet tall so you can see we have two different methods with the same name that is my class and followed by that the second my class and followed by that we have the same info or the same info method with different parameters now this is our main class which will access these two methods and print the data which we require now that we have gone through the program let’s execute this and see the output as you can see the program has been successfully executed and the output has been printed here you can see that the provided height is zero so the building is been provided as 0 ft tall if we replace the height with another number like say 100 then the program will be executed again and the updated data will be printed here so that with static polymorphism and Method overloading followed by Static polymorphism and Method overloading we shall Advance into the second type of polymorphism which is none other than the dynamic polymorphism so Dynamic polymorphism is a process in which a call to an overridden method is resoled at runtime that’s why it is called runtime polymorphism or dynamic polymorphism method overriding is one of the ways to achieve dynamic polymorphism in any object-oriented programming language overriding is a feature that allows a subass or a child class to provide specific implementation of a method that is already provided by one of its super classes or parent classes now let us execute a sample program to understand Dynamic polymorphism and function overriding as you can see this particular example is based on method overriding or dynamic poly morphism here we have a parent class and a child class the parent class is the animal and the child class is the dog so inside the parent class we have a method called move and similarly with the same name the same method is also included in the child class dog so what exactly we doing in the main program is we’re trying to call the parent class method move as well as the child class method move in our main program so let’s see and execute the program and generate the output you can see the program has been successfully executed and both the methods are being called so inside the parent class animal the method move is been executed and animal scan move message is been printed successfully followed by that in the child class dog the move method is been called and the message dogs can walk and run message is been printed successfully so with this let’s move ahead and understand the next Topic in our discussion that is the advantages of dynamic polymorphism so the first advantage of dynamic polymorphism is support for method overriding Dynamic polymorphism allows Java to support overriding of methods which is central for runtime polymorphism followed by that the second advantages it allows subclasses to add it specific methods subclasses to define the specific implementation for the same so these are the few advantages of dynamic polymorphism let us move ahead and understand the different characteristics of polymorphism in Java in addition to these two main types of polymorphism in Java there are other characteristics in Java programming language that exhibit polymorphism like coercion polymorphic coercion deals with implicit type conversion done by the compiler to prevent type errors a typical example is seen as an integer and string concatenation as shown in the following example you can see string s strr is equals to string is equal to 2 so this example is based on coercion which will be an implicit type conversion done by the compiler followed by cotion we have operator overloading an operator or method overloading refers to a polymorphic characteristic of same symbol or operator having different meanings or different forms depending on the context for example the plus symbol is used for mathematical addition as well as string concatenation in either case only text that is the argument types determines the interpretation of the symbol the first plus symbol is used in string data type so the plus will be considered as a concatenated operator and followed by that the next place where the plus operator is used is in the integer data type so here the summation of both the numbers will be performed and the output will be as this so the first one is 22 and the second one is f followed by operator overloading we have polymorphic parameters polymorphic parameters allows the name of a parameter or method in a class to be associated with different types in the below example I have defined content as a string and later as an integer the Declaration of polymorphic parameters can lead to a problem known as variable hiding here the local Declaration of parameter always overrides the global Declaration of another parameter with the same name to solve this problem it is often advisable to use Global references such as this keyword to point to the global variables within the local context now with this let us move ahead into the next topic that is the super keyword super is a keyword it is used inside the subass method definition to call a method defined in the super class let us execute a sample program to understand the functionality of super keyword as you can see this particular example is based on super keyword so here you can see we are trying to call a method from the parent class or super class using the Super Key word now let’s try to execute this program and see the output you can see that the program has been successfully executed and the method from the super class has been extended successfully into the child class using super keyword now with this we shall move ahead and understand the basic difference between the static polymorphism and dynamic polymorphism the first difference is static polymorphism relates to method overloading whereas the dynamic polymorphism relates to Method overriding the second difference is in the static polymorphism errors are resolved at compile time whereas in Dynamic polymorphism errors can only be determined at run time let us go through a code segment to identify errors in static overriding you can see the first line of code which is void difference between the numbers X and Y followed by that we have another method with the same name where we’ll differentiate the numbers X and Y of float data type and the last int diff A and B so here we identify a compile time error similarly let’s go through a code segment for dynamic polymorphism you can see the college here is the parent class and the object we creating is from the child class so the reference of parent is pointing to the child here that is college obj is equals to new student followed by that the method of child is being called which is obj do exam so this is an example for dynamic polymorphism exception is a problem that arises during the execution of a program it can occur for many different reasons say a user has entered an invalid data or a file that needs to be open cannot be found or you can also say that a network connection that has been lost in the middle of Communications or the jvm has run out of memory many such cases but if we do not handle them it leads to a system failure so handling an exception is very important and that’s where Java introduced exception handling mechanism to handle the runtime errors so that the normal execution flow will not be disrupted for example class not found IO exception SQL exception Etc now let’s move ahead and see the difference between error and exception first errors are impossible to recover but exception can be recovered by handling them next errors are of type unchecked but exceptions can be either checked or unchecked type just wait a little while and you will know what are checked and unchecked type of exceptions and errors are something that happen at runtime but exception can happen either at compile time or runtime exceptions are caused by application itself whereas errors are caused by the environment on which the application is running so these are the key points of difference between errors and exceptions now let’s see what is exception hierarchy all exceptions and errors types are subclasses of class throwable which is a base class of hierarchy here one branch is heeded by exception that is this class is used for exceptional conditions that user program should catch for example say null pointer X exception runtime exception Etc and other Branch error are used by the Java runtime system to indicate the errors that has to do with the runtime environment itself that is JRE for example virtual machine error or stack Overflow error Etc so how Java virtual machine handles exceptions whenever inside a method if an exception has occurred then the method creates an object known as exception object and hands it off to the runtime system system and this exception object contains name and description of the exception and also the current state of the program where exception has occurred and creating the exception object and handling it to the runtime system is called throwing an exception then using try catch finally methods these exceptions can be handled so this is how Java virtual machine handles exceptions internally next let’s see the differences between checked and unchecked exceptions and simp terms exceptions that happen at compiled time are checked exceptions like IO exceptions Etc and runtime exceptions are called unchecked exceptions like divide by zero array out of bound null pointed exception Etc basically unchecked exceptions are built-in exceptions in Java so checked exceptions cannot be simply ignored the programmer should handle these exceptions and runtime exceptions are ignored at the time of compilation now let’s let’s see the basic example of exception so here I have defined the class and inside the main method I have a tri block so in this tribe block the code that we are going to write May raise a exception that is it contains the code that raise exception and then the raised exception will be handled in the catch block so let’s see a small program as how it occurs first I’ll create a new package called exception edura and now I’ll create a new class called sample exception finish so inside the class the first thing that we are going to do is we are going to write the main method so here I’ll will create a string s Str and make it as null so now after this I will try to retrieve the length of the string so when I execute this program it will throw an exception why because the string is null and we are trying to retrieve the length of the string as I said it throws null pointer exception because the string is null it does not contain any values so now let’s use try and catch method to handle this exception so what I’m going to do here in Tri block I’m going to write the code that will raise an exception say I have given a 30 and B a zero now I will try to divide a number by zero like in C is equal to a by B so just tell me what happens here now say if we try to divide a number by zero then it says we cannot divide the number by zero because it raise an arithmetic exception correct so let’s see how that occurs now as it throws an exception here so what I’ll do to maintain the normal flow of execution I will write the cat block to handle the exception that is okay let’s run the program and see the output so as I told the exception has being catched by catch block so here first it will throw the exception and then it goes to the catch block catch block handles the arithmetic exception e so it prints cannot divide a number by zero so this is how exception handling works now let’s see the various types of exceptions so in Java there are two types of exceptions one buil-in exceptions and the other one user defined exceptions first let’s see what is buil-in exceptions these are the exceptions which are available in Java libraries and they are suitable to explain certain error situations like arithmetic exception array index outof bound exception class not found IO exception runtime exception number format Etc so let’s see an example and understand how built-in exception works so this example is one type of exception that is arithmetic exception where we try to divide the number by zero and it results in arithmetic exception and we have catched the method so let’s see one more example of that kind now I’ll use one more Tri block here what I’m going to do is I’m going to declare the integer number and try to pass a string variable so what happens it’s obvious fact that it throws an exception so let’s see how so just tell me one thing I have declared a integer number and trying to pass a string variable so what happens it obviously throws the exception now in order to maintain the normal flow we have to handle this exception so I’ll use a catch block and handle the number format exception so now let’s execute and see first it said cannot divide by zero because we handle arithmetic exception and next we handled number format exception next we will see how array index outof bound exception occurs and how it can be handled so here you can see I have created an array of size five and now I’ll try to access the seventh element in the array so what happens after this it will obviously throw an exception why because the size of the array is five and I’m trying to access the array that is out of bound so what happens when I catch the exception it says array index out of bound exception sorry it is array index out of bounds now again let’s execute as I said again the array index out of bounds exception got handled and it printed the output so this is how built-in exceptions are present in Java libraries so now I will jump into the methods of exceptions because you need to know the throw and throws methods to understand user defined exceptions after I complete the methods of exceptions I will explain you the user defined exceptions so these are the various methods of exceptions that is try catch finally throw and throws let’s see each of them first try try block is used to enclose the code that may throw exception the syntax is as follows as you have already seen it’s very simple we write a code that throws exception and we can handle it using the catch block next Nester try Nester try block is nothing but a try within a tri block for example you can see here I have declared one try and inside that I have created one more try catch and there is one one more try as well so let’s see a small example as how it works so I’ll do one thing here I’ll give a triy block within the tri block and then after catch I will end the statement and after that I will print like system. out. print Ln let’s print some other statement just look at the program first what I did so from here I have modified the program I have given one Tri block and inside one try block I have given one more try and catch block okay and now I have created one more try catch and handled here after that what I’m doing I’m just giving a statement as print other statement after that I’m closing this dry block then I’m again catching the exception and handling that okay so within this one Tri block there are two Tri blocks so that’s how Nest TR can be used so let’s see the output see first it’s saying that cannot divide by zero because it was the first thing that we have done and within the tri block it said number format exception because the first Tri block got executed and next for this one more exception that it WR it said it handled and again it’s printed other statement that’s all because we are not declaring anything in the other Tri block that is the main Tri block so it is not printing that exception if this drive block has thrown any exception again that would have been printed but as there is nothing it’s not printing so this is a usage of Nest a try and how it can be declared and this is how it works now let’s see the usage of catch block as in these previous examples you have already seen how it can be used and there’s also one more thing that is multic catch that is if you have to perform various task at the occurrence of various exceptions then you can use Java multic catch block so here you can see you have one Tri block and you have multiple catch blocks so let’s see how that works I’ll modify this program only for you so here I have one Tri block and I have two different catch blocks let’s execute and see so first it printed number format exception and then it printed other statement why this exception did not get handled because there was no any exception that was being thrown for this okay if there was any exception that was thrown then it would have been handled so this is how you can use multiple catch blocks next we’ll see finally finally is a block that is used to execute important code such as Closing Connection stream Etc no matter whether exception is handled or not finally block will always be executed so in the previous example you saw that after declaring a tri block you must use catch block to maintain the normal flow but on using finally block the catch block is not required let’s see how I’ll do one thing I’ll remove all the catch blocks here so I just have a tri block now I’ll give finally and print the statement okay so now let’s execute this as I said we have not used catch block it throws the exception but even on throwing the exception the flow will not be disrupted there because we have used finally block to execute the code so you can see the finally block is always executed so that’s how it works next let’s see what is throw throw is nothing but a keyword which is used to throw an exception so you can see the syntax here inside a method you are throwing an exception let’s see how it throw was an exception let’s modify this program first I’ll Define the function here so inside the throw block I’m going to throw an arithmetic exception okay now let’s handle the exception using catch block I’ll close this average function and call this function in the main method so first I throw the exception in tribe block and catched it and now I’m calling the entire function in the main method so let’s see what will be the output it’s just telling the exception is being caught correct why because when we call the average function in the main method it goes back to the loop exception will be throw but it is been catched correct so this is how throw is used to throw an exception and it can be handled the cach block simple let’s see what is throws throws is also a keyword which is used to declare the exceptions it does not throw any exception but it specifies that there may occur any exception in the method and it is always used with the method signature let’s see how again I’ll modify this program itself as I have told it is used with method signature I’m going to use the throws keyword along with the method and I’m also using a throw keyword to throw the arithmetic exception now inside the main method I’ll use a tri block so this is a example where in I’m using the throw keywords along with the method signature and inside this I’m throwing a new exception but when I’m calling the method or a function in inside the tri Block in main method it will throw an exception so instead of handling it I’m just printing the finally block so let’s see what happens so you can see here first the inside average function is printed and then it says that there is a exception in the main method that is we are throwing an exception as we are not handling that at the end it’s printing the finally block simple so this is how throws is used next let’s see the difference between throw and throws first throw is explicitly used to throw an exception but throw is used to declare an exception we have already seen how it worked next checked exceptions cannot be propagated using only throw but checked exceptions can be propagated using throws throw is followed by an instance and throws is followed by a class and throw is always used within a method and throws is used with the method signature and throw can throw only one exception and not multiple exceptions and throws can declare multiple exceptions just how we saw how it worked next we’ll see the difference between final finally and finalize so final is a keyword finally is a block as we saw and finalize is a method final is used to apply restrictions on class methods and variables but finally is used to place an important code and finalize is used to perform clean up processing just before the object is garbage collected coming to final the final class cannot be inherited and the final method cannot be overridden and the final variable cannot be changed coming to finally it will be executed whether the exception is handled or not we already saw how it worked now let’s see a user defined exception sometimes the buil-in exceptions in Java are not able to describe a certain situation in such cases a user can also create exceptions and that are called as user defined Exceptions there are two important points first one a user defined exception must extend the exception class the second one the exception is thrown using a throw keyword so let’s see how it works I’ll modify the same program first inside the tri block I’ll throw an exception using throw keyword I will handle the exception using catch block I’ll close the class now I will extend the exception using extends class so let’s par some argument here and let’s try to print the output so let’s par some argument as we have thr a new exception and when I’m extending my exception then what happens let’s C I’ve saved the program and I’m giving the value of a to B okay now let’s see what will be the output as expected it should print the value as five let’s see whether it prints the same prescribed value or not exactly we got the output as desired that is the throw keyword is used to create a new exception and throw it to the catch block and then it got handled and when we and when extending that we will get the same desired output so this is how how you can create a user defined exception now coming to the end of the discussion let’s see what are custom exceptions and how to pass an exception I’ll modify the same program again so in this case if the number that we entered is more than 100 then it should throw an exception else it should print the provided input is valid let’s see how after that I’m reading a number here and then trying to validate the input and tri block if the input is not validated then it will be handled in the catch block again this is a user defined exception so it must extend the exception class so this is how I have created custom exception so let’s run the program and check what will be the output so it’s asking to enter the number that is less than than 100 so I’ll enter 99 says valid input again I’ll run the program now I’ll enter 101 it says input is greater than 100 and it C the exception that is I have mentioned here if the number is greater than 100 then it should throw an exception and that exception is caught in the catch block so this is how custom exception can be used in Java now let’s also see how to pass an exception in this case we are going to pass a simple date format so for that I need a java. simple date format package I’ll import that now I’ll make the class as public and let’s write the code for the simple date format I’m going to create a reference called SDF so I’m going going to pass the input date that we enter so this is very simple I’ll tell you how if we enter the date in this format then it should print as valid input or else it should throw an exception let’s see how to do that so in the tri block first I have read the input date and then output date and have created a reference for them and if there is any error in case of changing the date format or while converting the date formats then that exception will be handled in the catch block and next in the main method I have used a scanner class to read the format so let’s run the program and check the output so first let’s enter the date format which is invalid so I’ll give something so it’s telling that some error has occurred while converting the date formats so it’s telling exception is here that is when we’re trying to pass the exception and the unpassable date will be so and so now let’s execute it one more time and enter the proper date format that is 0 1 01 1 996 so it’s telling after changing the date format it will be this that is the one which I have given in this format from the input date format which is in day month and year it change the output date format by passing the input date to year month and date correct so this is how we can pass the exception so to begin with guys array is a data structure to hold you know more than one values so it’s going to be a multivalue container for us so it can contain homogeneous elements so you can say it’s a limitation or it’s a plus point right so it maintains the type safety check so we cannot store different type of data in the same array so it has to be the same type so if you’re going to create an array for integers it must be only integers now all the elements they are stored under one single variable name so we have a reference variable which will be pointing to the array now the data stored in an array is a contiguous memory location it means that they are the neighbor now so there are various different types by which we can represent the data in arrays so we can represent the data as single dimensional or 1D array which is storing the elements as in a row so just consider it as a row in a continuous row we are storing the elements so that is how we started right so in a use case to store the country’s population so this element can store State one population count next element can store the other state population count so on and so forth now two-dimensional array is array of arrays right so it’s a collection of collection now so here what we are understanding two dimensional array that it’s going to hold the population for the world let’s say the first array will store the population for Indian States the second array can hold the population for some other countries different states so if you have more than these kind of problems so where you have data coming up in the form of some let’s say you want to process images right you want to rotate an image you want to process an image on some colors so here we just don’t have a two Dimension array we just not have array of arrays so we got multi-dimensional arrays so in that regard we got this multi-dimensional arrays where lot of

    data can be stored so typically when you are going to come up with the image processing problem so you’ll be working in the multi-dimensional array so one of the use cases now let’s see how we can work with arrays so guys how we can create a 1D array so creating a 1D array is very basic so you give a reference variable name and you mention the data type with the new operator with the size and you’ll be able to get your array created in the ram area so reference variable with the new integer five will give you an array of five integers right so you need to be having this syntax where we can create an array now when you’ll create an array it will be indexed on the basis of size let’s say the size is five so there will be four indexes so 0 to n minus one so it going to be 0 to four for a size of five now every element in an array is accessible through this index for example when you say my array 0 isign 10 you writing the data in the zero index so thereafter you say 1 is 20 2 is 30 3 is 40 4 is 50 so you are actually writing the data into the array so a basic right operation and same way when you have the array constructed so at the time of construction you can mention the data right so the values can be given exactly at the time of construction so this time you need not to write the data so by default we’ll have the values coming in for your array so guys when we want to access any element in an array so you write down the reference variable name and you mention the index in a square bracket and you will be able to access it for example you say my array one it’s going to access the value 20 so you say my array 4 right so array four over here is supposed to be the value 50 so when you want to update something so you can just mention the index and you can give a new value so here it goes like my array 4 assign 100 so you are going to replace your array 4 as 100 so let’s have a quick practical example on 1D arrays so what I’m going to do is I’m going to create a new Java project project so let’s say a new Java project called array demo so we’ll begin with uh why we need arrays and thereafter we’ll process it further right so I’m going to say arrays with the main mathod let’s say 1D arrays the package goes like cod. edura now consider you want to store the population of a country right so you will have state one let’s say the population count is 52 same way you will have some other states coming in let’s say State 2 state three state four and state 5 right so let us see some populations are coming in for us now having this population stored in five different single value containers right so what are they they are single value containers they cannot hold more than one value so when I’m talking about State one state 2 state three state 4 and state 5 they are the different states right containing the population so I can say State one population count to be more precise right so considering that these are the different population counts for the different states of a country so rather than containing these single value containers what is a solution is to have a multi- value container right so how you’ll create a multivalue container so we’ll write one integer array so I’ll say population I’ll just write down this name as state population which goes like a new integer with five elements right so I gave the size myself that this array should have five elements so when you print out this state population when you run this code here so you will see that we getting some hash code in a hexa decimal format it says some integer array at the 785 e922 so what exactly has happened here so guys in the ram area so whenever you going to run the program so in this Ram structure so you’ll have this stack coming in as a data structure and the second thing is keep for us so I’ll just quickly write so this is Ram allocated this is the stack data structure this is the Heap data structure now we got this guy called population so if you see this state population so I’ll not use the term state population rather I’ll say country population so this guy contains 785 e922 right so country population is containing some hexa decimal over here 7852 e922 so it can be any number which is coming for us over here because it is not generated by us now in the Heap area there will be a storage container which will be created at the same hex code right so this is key value pair I mean to say it’s like hashing as an algorithm so this guy acts as a key where this is the storage container so we got these indexes coming in for your array right so that’s like zero index one index two index three index and the fourth index and by default we get the data as zeros right so we have all the data coming in as zeros so we have everything over here as zero and this guy country population is basically pointing out here now coming here I will first of all read the element in Array the first thing is reading the element in Array so what I’ll do I’ll simply say please print me the country population zero element so when I’ll say zeroth element so here we are with the zeroth element right so when you run this code you get to see that the value is zero so same way if I read it for the first element so you will see it’s going to give me again zero right so this is like reading the element and it’s like read single element so you access the element right so there can be the other way around that’s like read all so I can just put up a for loop I can say start with zero go till country population do length so length will give me the size of the array so what I will do is I will say ciso country population of I is plus country population I so where each and every index will represent a state okay so now when you run this code here you get to see all the values coming in as zero so when you create any array so all the elements will be by default zero so what I can do is so before I access my array I can do a right operation in Array right so WR operation so WR slash update operation so both of them are same for us so I can come here and say that the country population of zero index is 502 same way we have one then the two then the three then the four so you got 765 9987 4 56 and 876 so we are able to write the data and now I’m just going to do an empty line here and we’ll say rereading after write operation so when you run this code here so what you see is once we do a reread operation after writing the content in the array so we see the updated data right so now we have these values coming in 502 765 987 456 and 876 it means that here we don’t have zeros now right so rather than zero so you got the data coming in for us for example this guy will go as 502 now every index represents a state with the population count right so so having single value containers differently so we are having single value containers differently rather than having it in in an array is a more better version now the same syntax over here can be replaced like this so I can say in Array country population goes like this number 502 comma 765 987 456 and 876 so what does this mean this means that so here by default all the elements were zero right so in this declaration when we read so all elements were by default zero but here elements have some default initial value so here elements they have a default initial value so even you don’t write the data so let’s say we are not writing the data here so when you try to read it so you’ll always have the data in the AR now right so we not performing rather any right operation so what you will see is that you have the data coming in before you even do right operation so choice is yours so whatever the way you want to write an array you can do that so you can have an array with all the elements having default value as zero or you just specify the values whatever you want to provide so the same syntax can come up like this as well all right so don’t get confused it’s the same story right so this is an implicit way versus this is an explicit way so choice is yours right so any syntax you want to follow you can follow that so now the next thing is uh update operation so you already have the data so you can use the same assignment operator to update the data right so let’s see this update operation coming into the picture so now when you run the code so before before you had this data and now you are having this data so guys I hope this is clear to everyone right so rereading after the right operation so now this right operation has become the update operation so we can use 1D array anytime with the help of indexes we can perform a read operation on a single element we can perform read on all the elements we we can even perform a write or update operation but everything happens with the help of indexing right so there is one more way to read the array in Java so that’s like your enhanced for Loop or sometimes we call it as for each Loop so any name can be given to this guy so you say for in any element in your country population array you say print me the element so you run the code here so you get to see the data once again so here you don’t have indexes this Loop will start from the zero index we’ll copy the element at zero index into this variable Elm which can be any name of your choice right so it’s just an identifier so we are trading in this Loop so now you seriously want to come up and calculate the population for this country so I can create a variable called count which goes like zero and here I can say count plus assign or let it be simpler count is Count plus the element right and let me come here and say population count for entire country is plus the count so let’s run this Cod here and what you see is the population count coming in now now you might be surprised in a way that this was a bit easier approach storing the values in a single value container and having State one population count so why we are messing around with these indexes even though this looks very easy now assuming that you have to work on 30 States so you’ll be doing 1 + 2 + 3 + 4 + 5 + 6 it’s like a lot of time consuming thing as in development effort is highly so having this one liner where you just mention all the population counts and having a loop as couple of lines and summing up the entire population so when data is having more Dimensions so you need to focus on array right so solving the problems where you have lot of data so you structure it in a better way and arrays they are one of the most easiest data structures to use in your software Solutions so with this let’s now move to the next part so the next part is working with two dimensional arrays right so guys we call them array of arrays right so here we mention the row and column so row column here does not literally means a matrix concept so row column over here means that how many number of 1D arrays you want so the first element means how many 1D arrays and the second means how many elements in 1D array right so it’s basically array of arrays we can represent a matrix using two dimensional arrays so this is like zero arrays zero element zero arrays first element same way first arrays zero element first arrays first element so if you want to access you can say 0 0 is 100 so zero arrays zero element right so you put the data so same way you can put the data for the other elements now if you want you can access any element in a 2d array by mentioning its indexes and if you want to update the data it’s again on the basis of indexing right so guys let us try to have one example on 2D arrays so I’m going to write one new class let’s say 2D array with the main method so to begin with I’ll just say in two brackets I’m going to say world population so considering this world population so I’m going to have the first array for the first country right so let’s say all right now this is first array and let it be the second one so this goes like some second array and this goes like some third array right so this is zero Index this is first index and this is second index now when we are understanding this problem statement so we mean to say that zero represents country one and one represents some country two so just just taking an example so this represents some country 3 same way here this is country 1’s zero country 1’s one one then two then three then four and five and six so this is zero State first state so these are the states now which we are talking about guys right States for country one so these are the States for country one right as in uh indexing so the next is the state for the country two states for the Country Three so these are all counts right so they are the population counts all right so let’s say that you got country one as India and let’s say the zero state is Punjab so we got the population count for Punjab so that is how we are representing it so there can be some other country two there can be some other country three so we have represented the data into this entire array now when I will just try to say world population is is plus world population and size is plus world population dot length or let’s say and length is now when I will run this code here as Java application so what you will see is that it’s an array of arrays at a hash code right and the length is three so I got this guy called world population so I’m just going to take this another example here so we got this guy as world population 785 e2922 some hash code so this world population is something like this so it’s basically having three indexes and you got indexes coming as 2 1 and 0o right so this hash code goes like this and we got reference coming in here now the next thing is we got the 0 1 and two we got this world population coming in now what I’m going to do is I’m going to access the zero Index right so I’ll say world population zero then one and then two right so we will come up here and say 0 1 and two so guys this is array of arrays right so zeroth index is an array first index is an array and second index is also an array so I’m going to say print me the Zero’s length print me first length and print me the second length so zero length is like what seven right because there are seven elements then the next guy and the next guy they have their own sizes so when you run this program here so you get to see the sizes as 7 five and six at the same time what you see is we got different different references coming in some 42705 C6 so we have some you know hash codes so I’m just going to represent these hash codes here let’s say some 40001 let’s say some 3001 let’s say some 2001 so I’m just considering these numbers as hash codes so further we got these arrays so this is like 4,01 array 3001 and 2 so guys I’m just considering these mathematical numbers as hash codes right so this is like further pointing out here this guy pointing out here and this guy pointing out here so what is this concept this concept is array of arrays so world’s population zeroth index is an array right so we get to see this data over here so this data is available over here right so as in indexes so zero Index right so this guy is indexed for us so zero index one index two index then three then four then five and then six so on and so forth guys the others as well so if you say world population zero arrays first element it’s going to be 4 5 6 7 so that is how if I come here and if I say please print me world population let’s say zero arrays zeroth element right so world population 0 0 you run this code here and what you get to see is 1 2 3 4 so guys I hope you understood this point right so when I say one and two so first arrays zero index first index and the second index that’s like 8765 so here you are right so this is like one and two so anytime you want to read the array so this is like read single element right so you want to write the data so write or update they work in the same way right so for example now I will be doing an update operation so I’ll say world population of one and two is some number all sevens so you say see so rereading and now when I will do a reread operation here so let’s run this code so here you are with the number all sevens so same way you can even perform a read all operation so I can write a loop so the outer loop shall work for the world population. length so this Loop’s going to iterate three times because the length is three so from 0 to two because we got three 1D arrays then I got an inner loop which will go from let’s say J as in zero we will say J less than world population of I do length and j++ right and thereafter I’m going to say world population of I arrays jth Index right the print line and print line we’ll do in the next case so let me give one empty line here so here you are so now you are able to print the entire 2D array so guys array of arrays they can be powerful data structures when you have lot of data so you can even create an array of arrays by saying an ARR is a new integer so I want three 1D arrays with five elements each right so three 1D arrays with five elements each so here if you’ll try to read this guy so if you try to read ARR so any guesses guys what will be the output when you read AR so what do you think what will be the output for the AR right so output for the AR would be all zeros right so when you give the sizes you get the default values as zero and when you don’t give the sizes you give the default values so it’s just the basic conclusion to understand the arrays right so giving sizes we need to put the data later hence data initially would be zero so this is how we are understanding this guy array right so giving data first we need not to give any size so that’s like the thumb Rule and guys explore enhanced for loop on 2D arrays right so explore that how you can convert this nested for loop with the help of enhanced for Loop right so that you guys explore so this is how we got something known as two dimensional arrays and let me do one more problem statement so how we can solve this matrices problem so if you can recall your childhood mathematics so we used to do an addition subtraction multiplication and lot of other things on arrays or on matrices so we’ll write one small program here and we’ll try to solve it as an addition problem right so let’s let’s perform an operation on array once again so let’s take a problem statement here to add two arrays so I’ll write one 2D array called a which goes like the first array having one one and one so I’ll have all the values as ones right so same way I’m going to have the other guy called B so this array B I’m going to have the values as twos so it’s a 3 cross three Matrix representation three rows and three columns three rows and three columns so when I have this metrix representation coming in so what I’m going to do is I’m going to print them first right so let’s see Matrix a so I’ll say for INT go with the zero go less than a do length and uh then I’ll write one more for Loop so we got this 2D array so we need to access the elements so you need to come up with a of I do length and j++ so here I’m going to print this guy called a of I and J so this is the representation for Matrix a let’s represent the Matrix B so I’ll say B do length B of I and B of i j so let’s run the code here so this is how the two different matrices are coming guys right now what I’ll do is I’ll come here and I’ll create one another Matrix that’s like C which is a new integer of three and three right so we’ll take three rows three columns and uh what I’ll do is I’ll take one Loop here run it till C’s length and I’ll say C of i j is a of i j so guys we are adding the elements here right and B of I so what we are doing is we are adding this one with this two then this one with this two so on and so forth right and once I’m done with it so I’ll represent this guy called Matrix C here after the addition so all elements they should come up as C so you run the code here so you get to see that the Matrix 3 is coming as all the threes so here we are so one basic mathematical operation on matrices guys right we’ll see sorting and searching in arrays so there are various different algorithms through which we can sort the elements in an array and same way we can even do searching now typically when you are in an e-commerce shopping website let’s say Amazon or any other e-commerce so you want to sort the items on the basis of the price right so that’s like either ascending way or the descending way so one of the practical usages where you can actually apply these sorting algorithms so these are some commonly used algorithms and they have their own you know time and space complexities but we are not going to discuss that in detail here so we’ll typically see how they work and how we going to solve this problem to sort the data in an array so we got bubble sort selection sort insertion sort quick sort and Mer sort as algorithms which we are going to discuss for the day so let’s begin with bubble sort so guys given the Sorting techniques so when it comes to the bubble sort it’s one of the simplest sorting algorithm right so what it’s going to do is it’s going to perform the swapping of the adjacent elements in case they are in a wrong order right so we going to always swap the adjacent elements if they are in a wrong order so we are considering that we going to sort the array elements in an ascending Arrangement so let us see how it’s going to work so consider that we got set of elements as 4 1 10 – 3 and 12 so what bubble sort will do so in case of bubble sort so we’ll start one of the iteration so consider the I iteration so we’ll start from the zero index and we’ll compare the index 0 with the index 0 + 1 that’s like 4 and 1 we are going to compare here so when we compare we find that 4 is greater than one so we swap the elements now the next is where we are comparing 4 and 10 so four and 10 will pass because four is lesser than 10 then we’ll move on to the next adjacent elements that’s like 10 and minus 3 so when we compared them we found that minus 3 is a smaller one so we swap them and same way when we compare 10 and 12 so it’s perfectly fine so let’s pass now in the next iteration so we’ll Again Begin and we’ll now start from the next index of the aat loop so here now we’ll see Min -3 and 4 has been swapped and thereafter we’ll compare 4 and 10 and finally guys the last iteration so on and so forth will finally sort the array so what we are doing we are just comparing the adjacent elements so if one is found to be lesser than the other so you swap them right so let’s try to write a small code snippet on bubble sort and see how it’s going to work for us so I’ll write one small program here so let’s create a new class so I’ll say a bubble sort with the main method so considering that we got the elements of an array so let us take the same use case so 41 10 – 3 and 12 so this is the input sequence right so this is something which we are are looking forward to sort now in order to sort I’ll write one method here so let’s say this goes like static void I’m just going to say sort here so I’ll take array as input so the array which I want to sort and here we’ll come up and say bubbl sort do sort the AR so you’re passing by reference it means that if you sort this reference right so automatically this will be Sol so it’s basically reference concept which we are using here so the first thing first so I’m going to now calculate the length of the array so let’s say length is er dot length so this is the size of AR mores right so we’ll run one Loop so this is let’s say the outer loop which starts from zero and this Loop shall go till less than your length minus one and and thereafter I’ll say an I ++ so it means that this Loop will go from zero so this I Loop’s going to work from the zero index then one then two and then three right not the last one right so it’s less than length minus one right so if the length of my array is five so let’s say the length is five so the outer loop here shall work from 0 to 3 so it’s length minus one so we want to iterate till here now I’ll have one more Loop so let’s say this is the J Loop so J Loops begin with zero and J Loop will run from 0 to length minus of I and minus of 1 and then I’m going to say j++ it means the jth loop is dependent on the I value so if the I value is zero consider that if I is zero so your J will vary from 0 to now length minus i – 1 right so what is the length length is 5 minus 0 minus one that’s like 0 to 4 so on and so forth it’s going to work for the other indexes of I now let us compare in case the AR’s J index is greater than arr’s J + 1 Index right so if if the element is greater we are checking for the adjacent elements now if the adjacent elements are being compared and we find that arj is the greater guy so we’ll swap it right and we’ll use the swapping technique as a very basic swapping technique so we’ll have a temporary variable in which I’m going to put this jth element and thereafter we’ll have AR’s G element allocated with the ARR J + 1 and lastly we’ll have ARR J + 1 having the 10 so I’ll just write down here swap the adjacent elements and this is how our Logics going to work right so we keep on swapping and once this entire swapping is done the loops are terminated so you will get one sorted arrangement of your elements So when you say bubble sort. sort the ER so now if you say for Loop let’s say int element in the AR so I’ll simply see so and I’ll say element with some space so you run the code here as Java application so what you see is the sorted arrangement of your array – 3 1 4 10 and 12 so this is the very first and the very basic technique called bubble sort so we compare the adjacent elements and we swap them so guys let’s move next so for our next sorting algorithm we got something known as selection sort so guys selection sort is also one of the Sorting algorithms and here this guy will sort an array by find finding the minimum element and putting it to the beginning so we’ll keep on repeatedly doing these steps we’ll find the minimum element repeatedly and we’ll just place it in the beginning of the array so that is how our selection sort is going to work right so let us understand this so when we are in the first iteration so we exactly are trying to see the minimum number and we we will find that minus 3 is the minimum number in the first iteration so we’ll put it in the beginning now we’ll shift the next index and we’ll just keep on comparing and we’ll see that one is the shortest one so let it be there so thereafter we’ll move the index to the next position and we’ll see now who is the other small element we’ll find that four is the small element will now come up and put it in the beginning and lastly we are in the sorted set right so selection sort is comparing the number so we are just going to compare the number with all the elements wherever we find the shortest element we put it to the beginning right so let us have one small program with respect to selection sort and see how it’s going to work for us so the way we got bubble sort I’m going to say a new class here let us say goes like selection s with the main method so we’ll take the same use case so instead of having this bubble sort will have selection sort so I’ll create a similar sort method so let me copy this so you’ll have a similar method where we’re going to sort the data right so the structure begins in the same way so the thing is we need to come up and iterate for the same structure we going to iterate from 0 to length minus one so the internal structure it’s going to be a bit different now right so what I’ll do is I’ll first of all assume that the minimum element or you can say that minimum element on the index is the element at the ath index right so the minimum element right so I’ll just write here so assuming minimum element is at ith index so we’re assuming this part now right and we’ll Now look for any element who is a minimum than this so for that I’ll put up a loop which starts as in J’s value with I + 1 and J shall run less than the length of the array so we are iterating over the entire array and if in case we say that ARR of J is less than ARR of minimum index so considering that what we have is we have the jth element less than the minimum index element so what I’m going to do is I’m going to say let minimum index go as J so in the array we are finding the minimum index and once we got the minimum element so minimum index over here means guys we are targeting the minimum element right so here we are targeting the values not the index so we are just capturing the index of the minimum element now once we captured that so there after what I’m going to do is I’ll swap the minimum element with the first element so swap minimum element with first element so what I’ll do now is I’ll take up one temporary variable in which I’m going to store the minimum index so thereafter let’s have AR of minimum index x with ARR of I and thereafter the AR of I will be having the temporary so this is like swapping the element and not the adjacent elements the way we had it in the bubble s so this is a bit different way of dealing with the things but at the same time when you will execute this program of yours so what you find is the data is again sorted for us so how these algorithms are different so they have the different approaches to solve the same problem so they vary in time and space complexity of course but that is not something which we are going to take now so guys this is something known as selection sort right so how we are selecting the element which is minimum and then putting it into the beginning now let us move to the next part so guys next part we’re going to come up and discuss something known as insertion sort now what is insertion sort so when I’m talking about insertion sort so this is very much similar when we are playing the cards so what we do is we pick up the smallest card and we put it in the beginning so here we are going to work in the same way right so we going to pick an element and insert into the sorted sequence so let’s see one iteration that how insertion sort will be working for us so for insertion sort let us assume the same input so when we will compare four and one so you see that over here we got four and one being compared so we’ll swap them and now we’ll move into the guy called 10 and thereafter we see that minus 3 is again a smaller element so we’re going to shuffle it to the beginning so it’s just like the way you sort your cards so finally you’ll get a sorted sequence right so guys insertion sort is something which is sorting the data in a different approach for us so here we are going to again find the smallest value and swap the same right let’s try to write a code snippet on insertion sord and see how it’s going to work for us so let me come here and create a new class so this goes like insertion sort with the main method so let me take the similar method here so we’ll call it sort itself and coming here so we are going to work from 0 to length that’s like 0 to 4 so we’ll take the same sequence and we’ll see how it’s going to work for us so I’m going to say insertion sort do sort s right so let us now write the code to sort the array now I’ll first of all consider that there is a key which is basically ARR of I so we’re assuming that the very first index itself is a key and I’m going to now create one J which shall be your I -1 so basically I want to move the elements of the ARR from zero to this J so that whatsoever are greater than the key to the position ahead of their current position right so let me write the code that’s going to be more helpful for us so here I’ll say while J is greater than and equal to zero and ARR of J is greater than T so run the loop till we have these conditions being satisfied and if the loop is in execution so what I’ll do is I’ll say ARR of J + 1 you just copy the ARR of J and you decrement the value of J by one and once this Loop is terminated come out of the loop and we’ll have J + one index with the key so that is how we are sorting the array over here guys so so in order to sort what we are doing over here is we are moving elements of ARR from 0 to i-1 right but we are moving only those Elements which are greater than key and where we are moving so we are moving them to one position ahead of this current position so if the current position is J we are moving it to J + one exactly the way we do it in the cards right so you’re just moving them one position so guys now let us try to run the same code and see what happens for the Sorting technique over here so what we see is insertion sort has also sorted the array and the technique is modified but the outcome Still Remains the Same so insertion sought in action so guys once now we have since understood what is insertion sort so we’ll be now moving into the other algorithm called quick sort quick sort is not as in that it’s really a quicker algorithm to sort the elements but when it Compares itself to the merge s yes it is quicker so what exactly happens in the quick sort algorithm so guys it’s a divide and conquer algorithm so what we do is we partition our array and then we try to solve the problem so consider the set of elements 9541 72 3 and six so what we try to do over here is we will decide one of the elements in this array which can be serving as a pivot or a pivot so the pivot element is the element which can be either the first element the last element or the middle element or any random element of your choice so it can be any element now the whole idea is to choose this pivate element and break your array in such a way that all the elements left to the pivot should be less than or equal to it and the right to the same pivot we should have greater than elements so we are going to make our pivot to go into the exact position so this way we will be coming up and sorting up the entire array so Shifting the pivot to the right position will be our key success now consider six is the pivate number from the entire list of sequence now consider the ath index or the ath variable and a jth variable so the variable J will be iterating over the array and we will be comparing each and every element to the pivot that’s like six so considering when your J is zero that’s like the number nine so we’re going to compare 6 and N so we know that when you compare the number six and N so 9 is greater than 6 right so we are okay to go with it now we increment the J and we bring it to the five when you compare five and six we find that this jth element is less than the pivot so whosoever is less it should move in the beginning or to the left so what we are going to do is if we find the element lesser than the pivot we are going to move it to the beginning or we are just going to swap number nine and number five and now I am again reiterating on the JF index and I have incremented the I index as well so let’s say initially I was pointing to this guy five but when we did a swap on 9 and five we incremented the variable I as well now the new number with the JF index is four so if you compare six and four so we understood that four is lesser than six so what we’ll be doing is we will be res swapping your number nine and four and then increment I now again six and one there compared and we see that one is less so you again swap but seven is greater you don’t swap but two is what which we know is lesser so what we’re going to do is we’re going to swap your two with the nine so on and so forth so lastly what you will have is you’ll have the elements that’s like 5 4 1 2 and three which are lesser than the six and 9 and seven greater than the six so we get this six exactly positioned after three and before 9 right so guys that is how we are doing this partition and then we are sorting the structure so let’s see a code snipp it on quick sort so I have this code walk through here so what I have is I have the input sequence of the same elements 95 41 72 3 and 6 so we got this sort method here where I’m passing array the zero index the length minus one so these are the initial inputs to this sort method so here we are with the sort method so if you got low less than the high which exactly it is so we are going to call the method partition with the array low and high now partition method is working in the same way so we are considering the pivot as the last index so high means the last index the last element now index of the smaller element shall be low minus one so that’s like we are trating this variable I and we are iterating in the jth loop which will iterate in the entire array so if the arj is less than the pivot we are going to swap AR I and arj and will also increment the variable I and finally once you are done with everything so what we we going to do is we’re going to move our pivot to the right place So eventually entire array to the left of pivot should be less or equal and to the right will be greater so putting the pivate to the correct position that is what we did in this partition method so recursively we are going to sort it and till time we don’t get the correct elements in the list we keep on sorting them so when we run this code quick sort. sort and you see we will have the elements in the correct order that’s like 1 2 3 4 5 6 7 and N so this is what we got As in quick s guys right so guys now let’s have look at the other algorithm for sorting that’s like the merge sort now merge sort is also a divide and conquer algorithmic approach so what we’re going to do is we’re going to divide the array in two halves right so you say it as left and the right one so here what we will be doing is we’ll take one example of the same sequence 4 1 10 – 3 and 12 so we going to divide this array into two different halves so once you divide the array so you keep on dividing the array further so you see once you are dividing the array you divide till we got all the single individual elements and once we have all the single individual elements right so therea we’ll start working on merging these arrays right so here guys will write something known as a merge function which will merge the two halves based on which is greater which is smaller so what we’re going to do is we’re going to merge first of all one and four we’ll see that one is smaller number so we’re going to place it first and then we’re going to bring it four right so so on and so forth we keep on merging the elements and we’ll first of all finish the merge on the left array then we’ll finish the merge on the right array and lastly we’re going to merge left and right array both which will be the sorted Arrangement at the end so in this merge sort algorithm divide the arrays into two subarrays left and right so thereafter you sort the arrays at the end right so let’s have a quick look on how this algorithm can work for us so I already have the code snipper for this so let me just open it so here we are so now typically for a merge sort algorithm how it’s going to work is so we got this function called merge and I’m going to take the same sequence of inputs so that’s like 4 1 10 – 3 and 12 so just give me a moment so 4 1 10 – 3 and 12 right so what we are doing here let’s see now so we got this guy called merge as one method which is supposed to merge to arrays and if you will see over here we got sort which is going to sort the two arrays and thereafter it’s going to merge so I’m going to initiate my program by executing the sort method right so for the sort method I am passing array to be sorted I’m passing the lower index that’s like zero and I’m passing the higher index that’s like your length minus one so this zero index and this guy is the fourth index now in case L is less than R which it is so we are going to come up and say let us find the mid L + r by 2 and we now calling the sort method twice the first sort happens on the AR r l comma M the second sort happens on M +1 and R right so once we sort the left and the right part then we are going to merge both of them so here when I’m doing a sorting so you need to understand that it’s a recursive call so we are calling the method again and again until this condition is satisfied so what happens in the merge part so here what we see is we got two different sizes right so S1 and S2 are the two different sizes for the arrays so we are creating the left array and the right array based on two different sizes so the left and the right array size so what I do is I copy the elements in the left array and in the right array so this is merge over here method so this is going to merge two arrays which we partitioned and you got l R which will be working from your L to M and we have this guy called let’s say R AR so there is subarray one and subarray two right so I’ll just say one and this guy ARR two so the second array is going to work from m + 1 till R now we are copying the elements here so create sub arrays so this is like create sub arrays and here what we are doing is we we are copying elements into sub arrays and now once you have copied the elements so we’ll start the process of merging right so let’s say start merging process here now in the merging process so what we got is we got indexes for the first and the second array so I’ll take the zeroth index to be the initial index and once I have taken that so I take the initial index of merged sub AR aray so that’s like K is L now while your I is less than the S1 and J is less than S2 so what we will be doing is we’ll be further checking if I index in the left is less than or equal to if it is so we going to assign the arr’s K index with the ith index in the left array and then you increment the I other way around it’s the right array which has to be considered so till I will be less than S1 we are going to come up and copy the ath index in the left array into the kth index and then you increment the I and the K so lastly guys if the J index is less than S2 you do it for the same part but you are doing it in the right array so we got something known as merge process and we got something known as sort process so when you run this code here So eventually you will again get the sorted outcome so guys sorting can work in various different ways so we got this technique called merge sort where you keep on dividing the array and thereafter once you have individual elements you start the merging sequence so this is the final outcome that how we going to work with it next part is how we are going to search in the arrays the first one is a linear search so what we going to do is we going to identify an element that whether it is there in the array or not so begin from the zero index will keep on advancing till the last index so once the element is found we get to know the index which index it is and thereafter we return back the index so in the linear search operation so we get to see that there are few inputs to the method so we going to return the index of the element which is found right so considering the use case right so we are planning to have this linear search so we’ll write one small program let’s have it with the main method so I’m going to take the same sequence of inputs which we have been following so this is the array and let’s say the element which you want to search is the number 10 whether the 10 is there or not so I’ll write down this static int search or let’s say this is uh linear search so in the linear search I’ll pass one array and element which I want to search right so this is how we are going to come up with the linear search approach so we’ll get to find the index so by default I’m keeping the value of index as minus one and I will return the index at the end now you just say for in I is z i less than the ARR dot length and I ++ right so you trading in the entire array so less than length is your n now if element is going to match the ARR of I right so if it is going to match the ARR of I so I’m going to say index is I and thereafter I’m going to break the loop so now you come here and say in found index is linear search dot linear search array and the element 10 we know where is the 10 element right the 10 element is at 0 1 and two second index so let us say if I is not equal to minus1 we’ll find the element and uh sorry so this guy is found index so let’s say if the the found index is not less than I mean not equal to minus1 so we got the element in the else case I’ll say C so element plus element not found or the other way around I’ll say C so element plus the element found at plus the found index so this is what we got so now when you run this code here so it say is element 10 found at two Index right so it’s the second index where you have this element 10 let’s say this guy as 12 you run the code it’s going to say at 4 index but if you try to enter something like 120 so you going to get element 120 not found so guys linear search algorithm is where you begin from the zero index and you keep on comparing till the last Index right so so the next searching technique over here with us is binary search so in the binary search what we’re going to do is we’re going to sort the array first right so we should have a sorted set of array and when you are going to find the element so you are dividing this array into two parts so we’re going to divide the array as in the left side and the right side so if you are finding some array in the binary search right right so this is now very much simple approach what we did in a linear search but the only thing is this we’ll be searching the element into a corresponding half so we keep on dividing the array to find the element okay so now let’s say you want to find this element number 79 so what we are going to do is we are going to take the right part of the array so as to search that element we will not into the lower half and into the upper half also we’ll keep on dividing the array again and again so let’s try to write a binary search algorithm and see how we can do a small search operation so this is basically going to save the time so this is optimized on time complexity so let me write a new class and this goes like binary search with the main method so similar to your for linear search I’ll just have this so this is what we got over here as let’s say static int binary search so I’m going to take array as input then I’m going to take something like left something like right and the last one is let’s say the element X so which you are going to uh search now the guy over here is going to check whether R is greater than or equal to l so if R is greater than equal to l then we going to find the mid so the mid goes like L + r – L IDE by 2 so we got the mid part and now so I’ll use recursive technique over here so in case so let’s say if element is found at the same middle index so I’ll just keep a check that if ARR of mid is equal to this guy called X then you just return this guy called mid so I’ll return the middle index itself right now there can be the other possible use cases so let’s say if element is smaller than the middle in case element is smaller than the middle so it means what hence element shall be present in the left part of array so element would be there in the left array and hence what I’ll do is I’ll just put up a small condition that in case AR RR of mid is greater than the X element which we are going to found so here I will again do the binary search so I’m going to do again the binary search but to do the binary search now so I’ll go with mid minus one and I’m finding the same guy X right so we finding the same guy X over here and lastly guys so I’m going to say return me so return me a binary search of er your mid + one right and X so else the element will only be available in the right part of array so this is is the case where element shall be present in the right part of array so if it is in the left so you execute this so return from here else uh you come if this condition fails we’re going to return this part and let’s say if we reach to a stage where element is not at all available right so if the element is not present then what then you should return – one so let’s try to come and say return of minus1 so here I’m going to perform a binary search called binary search and you need to make sure that the array is in a sorted Arrangement right so we’ll say this array over here so you need to sort this array first so what I can do is I can just make it Min – 3 1 4 10 and 12 right so that’s like the precondition array must be sorted first now when your array is sorted so let’s say you need to find the element called four and in the binary search you pass the ARR that’s like the first thing then you pass your zero index so guys let me just take some more inputs here so let’s say the length is ARR Dot length so this is the length of the AR and I’ll pass length minus1 and the element which I want to search so we are searching for this guy called four let’s give a small run on this and let’s try to see what happens so it says element 4 found at second index let’s search for 12 it says at Fourth index and when I say 120 you give a run here it says 120 KN pound so guys this is a binary search operation you keep on dividing your array into sub arrays to the left and to the right and then follow that array in which your element would be present right so greater than or less than condition check so this is a recursion approach through which we are solving this binary search problem so this is what we have over here as in the binary search part a Java collection framework provides an architecture to store and manipulate a group of objects and it includes interfaces classes and algorithm in this figure blue boxes refers to the different interfaces and the gray color rectangles defines the class so here list is an interface and array list link list Vector all these classes implements list as a interface similarly as d Q is also a interface so it extends an inherit Q interface now talking about array list it uses a dynamic array for storing the elements it inherits abstract list class and implements list interface then list interface extends collection and iterable interfaces in hierarchical order so this is about the hierarchy of array list class now let’s see what is array list array list is a part of collection framework and is the implementation of list interface where the elements can be D dynamically added or removed from the list also the size of the list is increased dynamically if the elements are added more than the initial size though it may be slower than the standard arrays but can be very helpful in programs where lots of manipulation in the array is required some key points to note here array list is initialized by a size however the size can increase if the collection grows or shrunk if the objects are removed from the collection next array list allows us to randomly access the list and array list cannot be used for primitive types like in care Etc to access all those we need a wrapper class for such cases now let’s move further and say the internal working of array list first we create an empty array and then go on adding the elements once the size of the array is full that is if the size of the current elements is greater than the maximum size of the array then we have to increase the size of the array but the size of the array cannot be increased dynamically so what happens internally is a new array is created and the old array is copied to the new array so by this automatically the size will be doubled and it will be increased so you can go on adding the elements how much you require so that’s how it works internally now let’s see the various Constructor supported by Java array list first array list this Constructor Builds an empty array list coming to syntax my array is a reference to an array list that holds the references to object of Type e array has has the initial capacity of 10 cells although the capacity will be increased as needed as references are added to the list and cells will contain references to objects of Type e next array list of collection C this Constructor is used to add all the elements of specified collection C to the current array list that is you can add all the specified collection C to the current array list next array list of incapacity here this is used to build an array list that has a specific ified initial capacity and initial capacity is the number of cells that the array list starts with it can expand Beyond this capacity if you add more elements now let’s see a small example to understand how and where these Constructors are used in Array list I will open my eclipse and first create package called edura now I will create a class called Constructor the very first thing that I do before writing the code is I will import util dostar package now let’s begin with the code I will first create the array list of type string I initialize the counter value to zero and using Advanced for Loop I’m trying to increase the count of counter but we don’t have any elements present in the array list so if I try to print the counter value it will not retain any value why because we don’t have any elements present in the list okay now I will create one more array list and initialize the capacity to that I’ll change the name and give it as B and I will initialize the capacity to 1 okay now again if we try to print the counter value it will not print because we have just initialized the capacity again if we try to obtain the value of counter it won’t get incremented why because we don’t have anything in the list now let’s create a string array and add some elements into it after creating a string array I will create a object of array list and then upend a element using ad method so let’s see how to do that so this is how I will add element using add method now I will use Advanced for Loop to print the elements present in the array list now let’s run the program and see what will be the output so here is the output as I have already told it won’t return the counter value because the array list is empty and the counter value will be automatically zero next on creating a string and adding the elements it has retrieved all the elements present in the string even with the one that we have added that is j2w so this is how Constructors are used in ARR list now let’s dive into the various methods supported by ARR list first add method this method is used to add the elements to the array list that is it is used to insert a specific element at a specific position index in a list for example you can create array list and go on adding the elements using the ad method we will demonstrate and see all the methods of array list in Eclipse after knowing the concepts of them next clear method this method is used to remove all the elements from the list that is you can just use Clear method to remove all the elements present in the list next trim to size this method trims the capacity of an array list instance to the list current size that is if you are creating an array list of size 9 and if you’re adding only three elements to that on calling this method trim to size it trims the size of array list from 9 to three that is it reduces the array list size to the number of element that it contains in the array list next index of this method Returns the index of first occurrence of the specified element in the list if that element is not present in the list then it returns minus one suppose in this case if you want to return the index of five and if you write here five then it will return a of two because a of one and a of two but if you try to find the index of three which is not present in the array list then it will return minus one that is it returns minus one if the list does not contain the particular element next object clone this method is used to return a copy of the array list that is on calling this method it clones entire array list so for example if you’re adding two elements in the array and you’re cloning the entire thing then again after clone list it will return the whole element present in the array list next object to array this method is used to return an array containing of all the elements in the list in the current order for example if you have all the elements in the list then it will return all the elements in the list in the correct order next remove method this method removes the first occurrence of a specified element from the list if it is present for example if you want to remove n from the added list then it removes the first occurrence of n if there is one more occurrence of n here it won’t remove that next size this Returns the number of elements in the list that is the size of the list suppose say we have added four elements then it Returns the size of the array list as four now let’s demonstrate all the methods and see how it works and how it is used I’ll create one more class called ARR list now in this example we’ll see how to add the elements we’ll create an AR list of type Al that is a reference and we’ll go on at adding the elements into this now we have added three elements now let’s try to print the size of the array and the contents present in the array let’s execute and see what will be the output so the size of the array list is three and the contents of array is edura Java and arrays now I want to remove Java from the list so what will I do here is I will use a remove method to remove the element from the list now again let’s print the size of the arist after deletions so when you remove Java from the list it retrieved only edura and aray and the size will be decreased to two so this is how we can remove the element we can add the elements and we can retrieve the size of the array list now let’s see how to clone the array list first we’ll make clone list as object and then call the method now let’s execute and see how the Clone list appears so as we have already removed the elements from the list so it it returns only the elements after deletion in the Clone list so if I make whole remove part as comment and execute then it will retrieve the entire elements present in the array list see that is elements in the Clone list are edura Java and arrays now let’s see how object to array method works on executing this this is how the output looks now let’s use Clear method and clear the array list it’s very simple Al do clear and print the elements after applying clear method nothing will be present in the array that is the entire list will be cleared now let’s see how to create a custom array list and add the elements move the elements and retrieve the size of that I’ll create a class called custom array say I want to create a class of student data wherein I’ll be initializing role number name marks and phone number so let’s see how to do that I’ll create a class called student data and initialize variables now I will create a Constructor and add the reference to the these variables now let’s begin with the main method now let’s create a custom array list and reference a variables using add values method I’m adding the values now I will invoke this function and Define it here I’m in working list of add method to add the elements in the list now in order to print the values present in the list I’m invoking a function called print values of list and here I will give the definition for that and now let’s print the values present in the list now let’s execute and see what will be the output that is it displays all the elements present in the array list that is the rle number name marks and mobile number now if you want to remove one of the rows from the array list let’s see how to do it using remove method here I will give list. remove now if you want to remove the row of elements from the array list then you can remove the elements from the list and then print the rest of the elements so let’s see what will be the output so output is like this first it has printed all the elements present in the list then it printed the size of the list and it removed the first occurrence when you use remove method it removes the first occurrence present in the list that is it removes the first value so it has removed the first row that is Chris and it retrieved the the rest of the elements so that’s how you can create a custom AR list and add the elements accordingly now let’s jump into the last topic of today’s discussion that is advantages of array list over array first array list is variable length arrays are a fixed length you cannot change the size of the array once they are created but array list is variable length that is it can grow and Shrink dynamically next size of the array can be modified dynamically when you add the elements into an array the size will be increased and if you remove the elements the size will be automatically decreased next you can add any type of the data Maybe list Union structure Etc not only that it also allows you to add the duplicate elements in the list next you can Traverse an array list in both the directions that is forward and backward directions using list iterator and it also allows you to insert and remove the elements at a particular position so I hope you understood the concept of array list and the difference between array and array list now if you’re talking about Java Springs right so we can process our textual data very easily given a use case so we write a sentence so we can very easily search some words in the sentence we can use an API like contains which can tell whether the word is there in the string or not right so so searching is one of the inbuilt feature in the string API then we can divide our string into you know multiple parts so we can say as a sub string moving ahead we can even create new strings out of the old strings for example the quick brown fox jumps over the lazy dog we can say the quick fox jumps and the quick lazy dog so these are the different strings which we can manipulate our existing strings and we can create the new strings but remember the old string never ever changes it’s always the new string which is created since strings in Java they are immutable now java. lang. string is a class in Java so guys this class is basically implementing few interfaces serializable comparable and Cas sequence so when we talk about the class string it’s a final class which you cannot inherit so every class in Java will be the child of object either you create or it’s a built-in class so it implements serializable comparable and the car sequence so let us see what these three interfaces will be for Strings now serializable is just a marker interface now when we talk about a marker interface guys so it will not contain any data member or method it is basically uh marking the Java classes so that our compiler can come to know about these classes now serializable are the ones for whom we can save the state in some files so there’s a concept called serialization and deserialization so saving the state of an object and recovering the state of an object from a file now comparable is basically that if you have a string and if you want to order uh the data in the string you want to sort you want to compare so we got some methods available with the help of which we can very easily do that for example a method called compare to can compare one string with the other string whether they are equal or not then we got cash sequence interface guys this is a readable sequence of correctors and this is supposed to provide some readon exess and the other thing is that car sequence is basically parent to three classes right string string buffer and string Builder now string is a class which can create textual content for us where this guy is immutable so we cannot change the data once

    it is created so you can even say strings they are nothing but their data or their values they are literals and we cannot change them but string buffer and string Builder they are again the car sequences which can manipulate the data within so they are the mutable versions of the strength moving ahead buffer and Builder they are different in a way that buffer is not thread safe whereas Builder is thread safe so my bad buffer is is thread safe whereas Builder is not thread safe right so what is thread safety guys thread safety is synchronization when multiple threads they are going to access the same single object there we need synchronization so string buffer is builtin and it is by default synchronized but the Builder is not synchronized so that is the difference between buffer and Builder now we’ll understand how we can create strings in the map memory so strings in the memory they are created in an area called string pool now when we are talking about string pool so Java string pool refers to collection of strings which are stored in the Heap memory so in the Heap area so there would be a string pool which will be created and as string objects are immutable in nature the concept of string pool came into the picture so string pool will help in Saving the space for Java run type so let us see how string pool is going to work for us and how it’s going to save memory for us now if you want to create a string there are two ways either you can create it as a literal we call it an intern string or you create it with the new keyword we call it an object right so let us have a quick look on how we create a string and how he and how the string pool comes into action so considering we are going to use little L Al to create a string so guys Java string literal is created by just writing the string literal in double codes so how you will do it you’ll say string Str Str is urea you need to use these double codes so when I’m saying St Str so guys St Str is basically a reference variable which is pointing to edura like this so in the string pool we got a literal called edura and Str Str will contain the hash code for that literal and uh this is how we got a relationship mapping now same way when we want to create a new string with the same name let’s say I want to write another string but that will also be Ura so for that what happens so before creating a string literal right so first a lookup happens for the value in the string pool if the same value is already label in the string pool right so a new string will not be created rather the reference of that string will be returned for an instance if I write string s str1 is edura it means what will happen St str1 will also point to the same string so literals they are created once they are not recreated same way when I will say string St str2 is welcome now welcome is not there in the string pool so what will happen a new literal welcome will be created to which s str2 will point so once again a quick review string literals we also call them an interned way of creating the strings in turn way of creating the strings is where a string literal gets created into the string pool once a literal is created it will not be recreated so if you set that string as R is edura so we are not going to come up again for string Str str1 is edura so it’s going to point to the same string literal for a different string literal we’ll have a different you know memory region created so string s str2 is welcome will point to a different literal and hence the string pool so how it is saving the memory by creating literals once right so sharing of string literals because these literals they are the values and they can never ever be modified they’re kind of constants now the way 10 one Z is a literal it cannot be changed same way these are the literals they cannot be changed now uh the other way of creating a string is if you write a new keyword so whenever you’re going to use a new keyword so it is basically going to create a new object in the Heap M so for an instance if I say string s str3 is a new string urea what’s going to happen so there will be a new object created in the heat memory so it’s like the just the other objects which get created in the memory right so it’s always a new object it’s not going to point to the string pool now so guys this is the difference between how we can create a string literal and how we can create a string object so object construction happens with the new keyword and it is always a new object so even if I will say string St str4 is a new string with adura it will be a different object constructed in the memory even though the contents are same but new keyword will always create a new string object in the memory now we’ll see some string Methods so guys we got string class which is a built-in class and we will be exploring these uh methods what are the built-in methods we will also try to explore them practically but let’s first of all have a look on on the list so we got this method called Equals which will check whether the uh given string is equal to the other string or not so it’s comparing the contents in the string now equals ignore case will ignore the case right so whether the upper case or the lower case it won’t matter for us length Returns the length of a string then we can get the character at any index now string is a you know collection of characters so they are indexed internally from 0 to n minus one so Car Act you pass the index and you’ll get the character at that index from the string two uppercase Returns the string in the uppercase lower case will generate a lower case string then replace can replace an old character with the new character and thereafter a new string can be constructed so remember whenever we are manipulating the strings whether the intern strings or the non- intern strings we are always getting a new string now trim is a method which is going to uh remove the wide spaces from the beginning and the ending of the string then contains method will give a quick check so it’s going to search whether a string is existing in a string so part of string exists in a string or not two car array so we can get our strings converted to the character array by calling the method 2 C array then we got is empty to check whether the string is empty or not ends with so we can check that whether the string ends with a specified suffix for example you want to generate an application where you want to list all the images so you’re writing a file explorer app you want to read the files and you want to check whether they are extensions are jpg or PNG so we got ends with right then similar to the ends with we even have starts with then we got concat which can concatenate the two strings right so it’s like appending the strings and regenerating a new string so guys let’s have small demonstration on string Methods so I’m going to create a new Java project so let us name this as strings demo so here I’m going to write a class let’s say strings Java strings I’m going to say a cod. Ura as a package with the main method there are two ways to create a string so I will say string St Str one is hello right so this is known as intered way or we can say string literals so where are they stored they are stored in stored in string pool right now I will say string St R2 is also Hollow so guys St str1 and Str str2 they are two different reference variables but they are pointing to the same literal hello Str str1 and Str str2 reference variables pointing to the same lital so as we saw in our uh string pool example during the presentation part now if I’m going to compare these strings if I will say if Str str1 is equal to Str str2 so here I’m going to say s str1 equal to s str2 in the lse case I will say St str1 not equal to Str str2 so what are we comparing here we are not comparing content that is hello so what we are doing here so we are comparing hash codes of Str str1 and Str str2 hash codes in Str str1 and in Str str2 which will be same for the string Hollow now literally is created once right so Str str1 as a reference variable and Str str2 as a reference variable they are pointing to the same string hello so we comparing the references here so when I will run run this code what you’ll get to see is that they are equal now the other way around is uh where I’m going to say string Str str3 is a new string of hello so what is this this is known as object string object that is we used new operator so here string object is construct in Heap area with Hollow as its contents at the same time when I will say string s str4 is new string hello so guys what has happened St str1 St str3 and Str str4 are reference variables which are pointing to two different objects who whose content is same now that is something which we need to understand so we are basically having these two different strings two different reference variables which are pointing to two different objects whose content is same so when we are going to compare Str str3 and S str4 we are not comparing the content we are again comparing the hash codes in s Str three and Str Str 4 which will be different for the different for the string object containing hello so see there’s a difference now right so if you will compare three with the four so when I’ll compare three and four you will see it is not equal to so this is the first proof of concept and as a conclusion we understand equals to will compare reference variables that is Hash codes and not the real content that is hello so equal to is comparing the references it’s not comparing the content and we also saw that if we have a literal if we create a string literal so there we get to have the same reference for the same strength so s Tier 1 was compared but s tier 3 and S 4 were not compare so how we should compare right so how should we compare strings there are two ways the first way I’m going to write here that if Str str3 dot equals Str str4 so that’s like the equals method it’s going to return me either true or false I’m going to say Str str3 is equal to Str str4 and the else part where it is not equal to Str str4 now when you run the code you see it is equal so remember if you want to compare the strings right so that is the content within right so you want to compare the content within you never ever use equal to operator because equal to operators is working it is working only on the reference comparison where equals as a method is comparing the content within right now the equals method is one thing the other thing is compare to Method you can say if s str3 do compare to St str4 and if this gives me zero right so this is a comparison St str3 compared to s str4 in the else case I can just say s str3 not compared to S str4 so let us run this code here so compared to S str4 and is equal to Str str4 so this is what we got guys right as in the two different you know ifls structures how we can compare the strengths now let us explore some string apis so more on string apis so some string Methods to begin with let us create a spring called s Str which is John Jenny Jim Jack Joe so we got this string John Jenny Jim Jack and Joe and what I’m going to do here is so I will see the first thing we’ll get the length right so I’ll say in length is s Str do length so we can say ciso length of s Str is plus the length now when I run the code here you see it gives us the length as 28 so when I get the length as 28 now there are characters right so I can just come here and say St Str do Car at0 Plus bar + SD do carat length minus one so when you run the code so even before I run the code so I’ll just print out the S Str so let’s say SD R is plus Str Str so I’m just going to print the string so it says Str Str is John Jenny Jim Jack and Joe length is 28 the character at Zer is J and the character at length minus one that is 28 – 1 27 is e so uh string starts with the zero and goes till n minus one that’s like length minus one now considering I will say Str Str do2 uppercase I will say St Str Now or S Str after uppercase is plus s strr so we are very simply trying to say uh that I want to convert the string to uppercase but when you run this code here you see that string is not manipulated so here we see no changes right so why we didn’t see any change so reason is that strings are immutable so when you run this code here so you see no change string Remains the Same so what exactly is happening here so you are getting a new string let’s say S1 so you’re getting a new string generated and when you will say see so S1 is plus S1 so you run the code here you see that S1 is manipulated a new string is created so guys whenever we are talking about string manipulation string manipulation means a new string will be generated but old string will not be modified so that brings in the first understanding that strings are immutable they cannot be changed we cannot modify their content now the next part so the way we got two uppercase in a similar way we got to lower case so I can search in the string I can say if s str. contain John right or let’s say not the John let’s say Jim so Jim is somewhere in between so I’ll simply come here and say Jim is in the string you run the code now it says Jim is in the string so contains method will help us to check whether a string is there a substring is there in the string or not now considering you want to break the string into a small string we can use a substring method I can say string S2 is Str Str Dot substring from the index number five so this is 0 1 2 3 4 and five index is this space right so I will say ciso S2 is+ S2 you run the code now what you see is that uh the guy John is trimmed off from the space till the end we got a new string so this is known as a substring a substring you mention the index so till from 0 to 4 string will be removed and there onwards five onwards you get a new string so even I can uh say string S3 is s dot a substring you go from six to uh let us say 10 so let us see what we will get in S3 so I’ll say S3 is plus S3 so when I run this code you see it is saying J NN now what is six let us see here guys right so 0 1 2 3 4 and this five till here we got the five index so start with six six 7 8 and 9 so 10 not inclusive right right so what is this so starts with sixth index and till 9 Index right so it’s like that is less than 10 so that’s like sub stringing so we can have a string so we can even replace the characters here so what I can do is I can say string S4 is Str Str do replace the character J with the character K so remember always you will get a new string right so now if you see S4 is plus of S4 so if you run this Cod here you see k canny Kim KAC and Co so instead of John Jenny Jim Jack and Joe now the next part here I will uh try to now uh you know get all the characters from the string so I will create a car array CH ER R which is s Str do2 care array so I have just converted the entire string into the array of characters so I got the array of characters from the string now I can say for car CH in ch so putting up an enhanced for Loop so I’ll just do a print not the print line and I’m going to print the CH along with some space that is what I’m going to do so lastly I put an empty print line so when I run the code here so you see the string with the space coming in so this was read as an array of correct s so we got array of characters coming in now let us take one of the use cases where I want all these uh you know names pulled out of this string that is like St Str I want to pull all the names from this string so what I can do is I can say a string array so let us say St Str RR so we will say St str s Str do split on the basis of comma so when need to put in a pattern on the basis of which we can uh you know split the string so thereafter I will say for string s in s Str ER so I’m going to do a ciso on S so when you run the code here what you see is that we got all the strings coming in but here we can see some uh space in the beginning because we got a space in the beginning after this comma right so if you want to remove the space you can just call the method called trim so trim is going to remove the space in the front or in the end beginning or ending right so now you see there is no space so splitting will help us to get the strings out of uh the string on the basis of some pattern right some pattern now moving ahead I’m going to write a string called email which is like John at some example.com and string like let’s say phone which is some phone number let’s say 9 999 and thereafter 88 8 8 and 8 so some string right for the phone number and let’s say there is a password so for this uh the password goes like password 1 2 3 so uh this is what this is data during registration right so while we are registering the user we are going to have this data from the user so you want to validate the data right so we would like to validate the data from user that is whether the data is correct or not right so I can simply come up and say if email do is empty so I’ll put a not here so if email is not empty right so I I can say ciso email is available other than in the else case I can say ciso please provide an email now if email is not empty email is available there can be even one more check that if email do contains atate and email do contains a DOT operator so if it is containing then it’s a valid email right so if I’ll put a not here it means that if anything is missing I can simply say email is available but email is not a valid email in the else case I can see see so email is a valid one so this is how we can you know use our string operations to check whether we get the correct string from the user or not so when I run this code here it says email is available email is a valid one so if you don’t put the email so let’s say email is empty here and you run the code so what you see is that email over here is not provided so it says Please provide an email so same way if you write an email with missing of at the rate so it says email is available all right so I think here I just need to put a check email contains so it should contain this and this right so so email is available but the email is not a valid email so we can put some conditions and check whether the data is correctly formed or not at the same time I can simply come here if phone dot is empty so if phone is not empty right same way the else part so please provide a phone number so here what we can do is we can see phone number is available and I can put one more check if phone do length is let’s say uh not equal to 10 so we are looking for a phone with the length as 10 right and the else part so I can say C so phone number is a valid one right so here I can say but phone number is not a valid number so similar validations right it says phone number is available phone number is a valid one so you miss out the phone number here you run the code it says please provide a phone number so you can get the things in order guys right so if the length is not 10 it says phone number is not a valid number so we are putting some validations here so same way you can put it on password now all these discussions which we have done so we are concluding one thing that strings are immutable so it means what if I create a string let’s say string is a new string and I say hello here and thereafter I will say spring do concat a space and a world right so concat is to append so we can even append the data like uh with the plus operator so that will also pend the data now I will say ciso string is plus the string if you run the Cod here it is just hello so this brings up the conclusion that strings are immutable we cannot change them so it is always a new string which will be constructed if you’re are going to do a concatenation now we got a version called string buffer so here I’m going to say hello now buffer and strings they are similar so we we will use aend instead of concap right so here I’ll say world and now when you’ll say ciso buffer is plus the buffer so what you will see is that in case of string buffer string is appended so data is manipulated in case of string buffer but not in case of string over here so this brings out that in case you want to manipulate the data in the same string right so we got this guy called string buffer which is mutable so mutable sequences of the string they are known as string buffer as well string Builder so the way you got buffer in a similar way we have string Builder the only difference is that Builder is not thread save whereas buffer is a thread save structure so the output shall be same for both of them and the usage is also same so I will say Builder is plus the Builder here so you run the code so you see buffer and Builder they work same way the difference is that buffer is thread save whereas this guy is not thread save so guys all the three versions like string string buffer or string Builder so we are very well aware with the uh you see strings now so we also understood what we need to do in case we need to use the mutable sequences so all three they Implement car sequence it means the reference variable to the car sequence let’s say CS it is let’s say null so CS can point to a string let’s say hello CS can also o point to a string buffer let’s say awesome and CS can also point to a string Builder so that’s like bu so that’s like runtime polymorphism right so this is like runtime polymorphism inaction so this is what we got As in the strings where uh strings they are immutable if you want mutable versions you go with the buffer and the Builder guys we got more many more apis in the string string buffer and string Builder so Explore More from here what is an interface in simple words an interface can be defined as anything which shares a boundary between two components in case of computers it separates two systems and access a boundary which helps in data exchange now what is a Java interface a Java interface can be defined as an abstract type which stores the methods and helps other classes to implement the behavior of the methods which an interface actually stores now you might get a question we have interface for that yeah absolutely with this we begin our next question why we need interface we all know that Java supports inheritance but when it comes to multiple inheritance Java cannot support it it ends up facing ambiguity between two parent classes and fails to provide the required result this particular problem is called as Diamond problem here as you can see there is a super class and this super class is been inherited by two classes called as Class A and Class B now we are trying to inherit the properties of Class A and Class B into a new class which is Class C which practically seems impossible in terms of java now the question would be is it seriously impossible to do it that’s when the interface come to the picture using interface we can achieve multiple inheritance here we are inheriting one class and implementing the behavior of another class confused let me explain with a little example let us consider an airplane which requires both the properties of carrying huge cargo and passengers let us assume we have two planes one is capable only to carry passengers and the other one is capable only to carry cargo now we need to carry both passengers and Cargo in one single plane which seems to be impossible on the basis of how the Java works as it feels it cannot access the properties of two different parent classes at the same time but you can make it possible by making Java feel that it is inheriting one plane and implementing the methods present in the other one it is like building a commercial plane which takes both the passengers and cargo luggage interface is like making a bigger plane which could do both the tasks without interfering with the components of one another instead just borrowing the methods which are present in the interface class now let us get practical first let us understand what was the problem in multiple inheritance here I have defined the first class class A and then the next class which is Class B and finally the class C here I’m trying to inherit the properties of Class A and Class B together in the class C by using extend keyword as you can see the editor is throwing an error which says that it is not possible to inherit the properties of both the classes now let us find the solution for this the solution is none other than using an interface here in this solution I am creating an interface I’m having the same messages which I had in my earlier example both the messages present in class A and Class B and similarly I have also defined the interface which has both the methods hello and welcome now let us run this program and see how does it work as you can see it is accessing both the methods and printing the message present in both the classes Class A and Class B now let us try a different example in this example I am declaring four methods addition subtraction multiplication and division now let us enter into a class here I’ll be trying to access all the four methods and print the output now let us try to run this program and see how the output appears as you can see the program has been successfully compiled and now it is asking for two integer values to perform the first operation which is addition the output has been successfully generated and now it is asking for two more integer values to perform subtraction the difference is been displayed now it is asking for two more integer values for performing the multiplication operation the output is clear now let us perform our final operation which is division as you can see we have our result here now let us understand the interface nesting as you can see I have nested an interface by the name inner interface inside another interface which is the outer interface and then I’m trying to generate a Fibonacci series using the method which is present in the inner interface which is other than the inner method using this method we will be generating the Fibonacci series let’s try to run the program and see the output as you can see we have our FIB series printed here and this process was being executed from the nested inner interface method now let us try to see an example for nesting by nesting an interface inside a class here I have nested an interface called edureka interface in inside a public class which is edureka class and inside the edureka interface I have a method called nested method using which I’ll be performing a string reverse operation let us execute this program as you can see the string I have provided is edura and after the operation you can see the string has been successfully reversed now let us have a one short quick revision about how to declare an interface so this is the syntax for declaring an interface where you will be using an interface keyword followed by the name of your interface and inside the interface you can declare your methods by default all the methods which you declare in interface are considered to be public with this let us find out the few differences between class and an interface the first difference is multiple inheritance interface was designed to provide multiple inheritance where on the other hand classes cannot the next difference is data members interface does not have data members all it includes is methods which decrease the probability of confusion during the implementation process on the other hand classes include data members which mean to say that the user must be careful while using the data members to avoid ambiguity the next difference is Constructors interface does not have Constructors while on the other hand classes take up the advantage by including them which will help them to set the values to the members of an object and the next one is complete and incomplete members interfaces comprise of only methods which make them have only the signature methods in them while on the other hand classes include both data members which are also called as abstract members as well as methods which are known as signature members the next one is access modifiers interfaces does not have access modifiers by default interfaces take up public as their access modifiers whereas classes provide private access modifiers which are not available in the interface the next one is static members interface cannot have any static members whereas class has all its members as static with this let us get into the major advantages and disadvantages of interface first let us begin with advantages we can achieve multiple inheritance in Java we can easily break up complexity and enable clear dependency between the objects we can achieve Loosely coupled applications through an interface don’t just be on the bright side interfaces have some disadvantages as well Java interfaces make the application slower when compared to the competitors like python which support multiple inheritance the next one is Once An interface is included in an application it might be used once in a while or it might end up being used multiple times at a larger scale with this let us discuss about the key points about Java interfaces the first one is can we instantiate an interface well we cannot create an object of an interface hence we cannot provide instances in an interface the next one is abstraction the major key advantage of interface is abstraction because none of the methods declared in an interface have a body the next key point is implementing an interface the keyword Implement is used in a class to implement the methods of an interface the method in an interface must be provided with an access modifier as public by default the methods declared in an interface are also considered as public class must Implement all the methods declared in an interface or else it must be declared in an abstract class let us see an example about implementing an interface here I have defined an interface and declared two methods into the interface method one and Method two the first method is used to find out a square root of the given number and the second method is used to print a message now let us try to execute this program as you can see it is asking for an input now let us provide a number to find out its square root as you can see the square root of the given number is four now let us try to execute a different example where we’ll be implementing the methods from two different interfaces as you can see this is my first interface and here my first method is to find out whether a given number is an amstrong number or not and later let us see a second inter interace where I have declared a method to find out if the given number is prime number or not now let us try to execute this example and see if the program can Implement both the methods from two different interfaces or not as you can see the class edureka 2 has successfully implemented both the methods from interface one and interface 2 and displayed the output I have given three as the input for finding whether it is prime number or not and the result is it is a prime number similarly I have given 153 to find out if it is an Armstrong number or not and the program provided the output as it is an amstrong number with this let us continue with our key points the next one is access modifiers in an interface interface can be declared as private protected and transient all the interface methods by default are abstract and public variables declared in interface are public static and and final by default interface variables must be initialized at the time of the Declaration otherwise the compiler will throw an error inside any implementation class you cannot change the variables declared in an interface let us discuss each one of these points practically here I have defined an interface by name try here I’m trying to declare an integer type variable a using int a is equals to 10 which is valid and similarly you can do it by using an access modifier public which is same as the previous one and you can also declare this variable by using public access modifier and making it as static and final the variable can also be declared by using final keyword and also the static keyword all the above stand the same now let us see the next point which we discussed in our access modifiers we discussed that the variables which we declare in an interface must be assigned to a value here I’m trying to declare an integer type variable X without assigning it to any value now let us try to run this program and see if there is an output or not and I’m trying to access the same X here in the main class and let’s execute this program as you can see there is an error which means that we were supposed to assign a value to the variable which we declared in the interface with this let us move on to our next stage of key points the next one is extending an interface an interface can extend any number of interfaces but cannot Implement them whereas a class can Implement any number of interfaces if there are two or more same methods in different interfaces then the class can Implement all the interfaces but one single method is enough to perform the operation a class cannot Implement two interfaces that have methods with same name but different return return type the major advantage of using an interface is that variable name conflicts can be easily resolved by using the interface name let us see this practically as you can see here I’ve defined an interface a with the method display and similarly I have also defined an interface B with the same display method inside it now I’m trying to implement both the interfaces A and B in my class same and I’m trying to access the display method let us try to execute this program and see if this runs or not as you can see the program has been successfully compiled and the output has been generated which says displaying data now let us discuss the next Point here I’ve defined an interface a which has the variable X of integer data type storing the value thousand and similarly I have also defined another interface with name B which has the same variable x with storing different value now in my main program if I try to access the variable X then it might provide me an ambiguity but if I use the interface name then I can successfully access both the variables and display the data present in them without facing any of the ambiguity now let us execute this program and see the output as you can see the value of x in the first interface a was th000 which is displayed here and the value of x in the second interface which is 2,000 is also being displayed here I used the interface name a and interface name b in the print statement provided here a data structure is a way of storing and organizing the data in a computer so that it can be used efficiently it provides a means to manage large amounts of data efficiently and efficient data structures are key to designing efficient algorithms next we have two types of data structures that is linear and hierarchical so what is linear data structures linear data structures are those whose elements are sequential and ordered in a way so that there is only one first element and has only one next element there is only last element and has no previous element while all the other elements have a next and a previous element understood like you can have only first and the next element and there is only last element but has only one previous element so there cannot be any multiple previous elements or multiple next elements okay but all the other elements have a next and a previous element note that there is only one previous and next elements simple So based on these categories linear data structure are being divided into stack q and Link list now talking about the stack stack is an abstract data structure it is a collection of objects that are inserted and removed according to the last and first out principle that is Leo objects can be inserted into a stack at any point of time but only the most recently inserted that is the last object can be removed at any time so you can see here this is a bottom element and this is a top element when you push the element into the stack it goes into the bottom and the last element that is the top element can be popped out so once the top element is popped out the next element which is available in the stack can be popped out but the bottom element cannot be popped out first understood so this is how it works that is it is an ordered list in which insertion and deletion can be performed at only one end that is the top it is a recursive data structure with a pointer to its top element it supports two fundamental methods that is push that is you can insert the element to the top of the stack and pop remove and return the top element onto this stack so if I have to give you an example it can be reversing a word to check the correctness of a parenthesis sequence implementing back functionality in browsers and many more now let’s see a small example of Stack so I have created a class called stack and the maximum capacity of the stack is 1,000 and as I told we need an element called top right because it’s a main thing so that’s a reason and then I’m initializing the maximum size of Stack so there are two conditions first is empty so if the stack is empty then return top less than zero because there is no element in the top when stack itself is empty then there is nothing right that’s a reason and then I’m creating a Constructor for stack and initializing the top size to minus one because nothing is there next what I’m doing I’m pushing the element into the stack so if top is greater than equal to maximum size minus one then print stack Overflow return false else insert the element to the top of the stack and initialize it to the variable that is X which is this one next say that so this element which is the element that you have inserted into the stack is pushed into the stack okay then return then again I’m creating a method called pop that is removing the element out of the stack so if top is less than zero then say back underlow else int X is equal to delete one element from the stack that is a of top minus minus then return X again I’m creating one more method called Peak if top is less than zero then say stack underlow else return the element at the top of the stack and return send it to X that is the element which you have taken it over here simple now I’m writing a driver code that is the main method I’m initially izing a stack and creating a object of a new stack and sending elements push 10 push 20 and push 30 and then when all these elements are inserted into the stack if I want again I can you know say push 4 so when I use S do pop so what do you think will this element will be popped out first or this element obviously this element right so now let’s run and check for the output so you can see first 10 was pushed into the stack 20 was pushed 30 was pushed into the stack and then 40 but when I used s. pop the last element was popped out so if I don’t have this element I’ll just save this now see the last element which is 30 is popped out from the stack right so this is how you can execute your stack and these are the operations that is performed on a stack so this is how it works now talking about the next data structure that is q q is also another type of abstract data structure and like a stack the Q is a collection of objects that are inserted and removed according to First in first out principle that is elements can be inserted at any point of time but only the element that has been in the queue the longest can be removed at any time so you can see here it has two ends that is front end and back end that is the front front end and the rear end so if you insert an element it goes to the front and that can be popped out that is removed but in case of Stack it is last and first out in case of Q it is first and first out it supports two most fundamentals that is NQ and DQ NQ means you have to insert the element at e that is at the rare of the Q and DQ means remove and return the element from the front of the queue so whatever element you insert at the rare end will go to front as soon as you inserted more right so simple that can be taken off from the Q end so Q’s are used in the asynchronous transfer of data between two process that is CPU scheduling dis scheduling and other situations where resources are shared among the multiple users and served on first come first serve basis so these are some of the situations where Q can be used okay so now let’s take a an example and understand how it works so this is very simple I have created a class called q and I’m creating a string of Q and assigning it to a link list so what I’m doing is I’m using q. add and I’m adding all the four elements and then displaying the elements present in the Q so when I say q. remove of three it will display the elements present in the que and the Q size and says whether it contains the element two or not if yes it displays that let’s run and check the output so first when I print this it displays the size and let me tell you one thing this is a inbuilt function so that is the reason when I’m using add it is being added automatically so then what I did I removed the element three so you can see here I can remove any element it’s not that the element which is there at the last or the element that is there at the top or nothing like that you can just remove the element from any corner and then when I remove the element and I print the Q it says only the remaining elements that is 1 2 and four next Q size is Q do size that is three and then whether does it contain the element two in the Q yes true if I say element three in the que it should say false because I have popped it out right so let’s see what it says it says true because initially in the Q it was present so if I now say say element 5 then run so when I say Q do contains 5 it should say false because it’s not there in the list right see it’s telling false so this is how you can perform the operations on the Q the next data structure is link list a link list is a linear data structure with the collection of multiple nodes where each element stores its own data and a pointer to the location of the next element The Last Link in the link list points to null indicating the end of the chain element in the link list is called a node the first node is the head and the last node is a tail so there are three types of Link list one is singly link list that is unidirectional where it contains only the information of the next node so when it comes to W link list it contains the information of the next node also the pointer to a previous node and when it comes to Circular link list head points to a tail and the last node points to the first element of the first node that is the last element of the last node points to the first first element of the first note so if I have to tell an example so imagine a link list like a chain of paper clips and they link together you can easily add another paper clip to the top or bottom it’s even quick to insert one in the middle all you have to do is you have to just disconnect the chain at the middle add the new paper clip then reconnect the other half a link list is a similar way of doing all these operations let let’s take a small example and understand this in a better way so I have created a class called link list example and node head is a head of the list and I have created a class called Static node the inner class is made static so that main can access it that is the main method can access it so I have the data and the next node and the Constructor for the node is int D because data is equal to D and next should be assigned to null now the next step is to insert a new node so for that what I’m doing I’m creating a static method for linkless example and I’m inserting linkless example list and int data and I have to create a new node with the given data so what I’m doing I’m creating a new node with the data and assigning the next node to null because that is mandatory so if the link list is empty then make the new node as head so if link list is empty that is if list do head is equal to null then make the new node as head that is new node is equal to head head else Traverse till the last node that is node last is equal to list. head and then insert the new node again insert the new node at the last node and finally you have to return the list by head okay now this is a method to print the link list so current node is equal to list. head and the link list contains the elements that is being present in the current node and the list that is already being inserted so when I have to travel to the link list I’ll check whether current node is not equal to null if it is not equal to null then print the data at the current node using the statement or else go to the next node in the main method I’m creating a object of a new link list and I’m inserting the values into the list in sequence then finally I’m printing the list okay so now let’s run and check the output so see the elements are being inserted in the sequential order so this is how it works so this was all about the linear data structures now moving further we have hierarchical data structures which comprise of binary tree binary Heap and hashmap as well so binary tree is a hierarchical tree data structure in which each node has at most two children which are referred to as left child and the right child each binary tree has a group of nodes that is root node left sub tree and the right sub tree the root Noe is the topmost note and often referred to as main note because all the other nodes can be reached from the root and left sub tree and the right sub tree is also a binary tree and binary tree can be traversed in two ways that is depth first Travers veral that is in in order pre-order and post order and breadth first traversal that is level order traversal and the complexity is Big go of N and the maximum number of nodes at level L is 2 ra to the power L minus1 and binary search applications include it can be used in many search applications where data is constantly entering and leaving also as a workflow for compositing digital images for visual effects used in almost every high bandwidth router for storing router tables also used in Wireless networking and memory allocation also used in compression algorithms and many more so this is all about the binary tree now talking about the Heap it’s a complete binary tree which answers to the heat property in simple terms it is a variation of a binary tree with the property is like a tree is set to be complete if all its levels except the possibly the deepest are complete the property of the binary tree makes it suitable to be stored in an array it follows Heap property that is a binary Heap is either a Min Heap or Max Heap Min Heap is for every node in a heap node’s value is lesser than or equals to value of the children maximum binary Heap is for every node in a heap the noes value is greater than or equal to values of a children and popular applications of binary Heap include implementing efficient priority cues efficiently finding the K smallest elements in an array and many more the next concept is Hash tables imagine that you have an object and you want to assign a key to it to make searching very easy to store that key value pair you can use a simple array like a data structure where keys that is integers can be used directly as an index to store data values however in cases where the keys are too large and cannot be used directly as an index a technique called hashing is used in hashing the large keys are converted into small keys by using hash functions the values are then stored in a data structure called a hash table a hash table is a data structure that implements a dictionary ADP that is a structure that can map unique keys to values in general a hash table has two components one bucket array and the hash function a bucket array for a hash table is an array a of size n where each cell of a is thought of as a bucket that is the collection of key value pairs the integer n defines the capacity of the array next hash function it is any function that Maps each key K in our map to an integer in the range 0 comma n minus1 where n is the capacity of the bucket array for the table when we put objects into a hash table it is possible that different objects might have the same hash code and this concept is called as Collision to deal with a collision there are techniques like chining and open addressing so these are some of the most Basics and frequently used data structures in Java now that you are aware of these you can start implementing them in your Java programs so I hope you understood the concept of linear and hierarchical data structures what is link list a link list is considered as a data structure similar to any other data structure like arrays stacks and cubes the link List have three major parts head tail and node every node in the link list is interconnected using the address of the next node before we get started with the link list let us understand the terminology of array before using an array we actually reserve the memory blocks in the memory to store the data in order to process them for example I have reserved an array of integer data type and I’ll be storing five elements into it the name of my array is my array and the addresses for the array are 0 1 1 2 3 and 4 so now let us try to add the elements into my array the first element is 10 as you can see the first element is stored in the index zero of my array similarly I’ll try to add four more elements into my array which are 2030 40 and 50 as you can see the data is been successfully inserted and the insertion order is been secured each and every element is been inserted into the array in a sequential order and now we can access them using their index numbers but in case of Link list this doesn’t happen unlike arrays they do not work with the completely reserved sequential collection of memory instead they choose the random blocks from the Heap and store the data into it and provide access to the data whenever needed the best part is once the use of Link list is done and if you won’t need it then you can actually dump the memory blocks back into the memory Heap which makes them reusable once you provide the data into the link list it will borrow the particular number of memory blocks from the Heap based on the data typee you provide for example two or four bytes for each block of an integer data type and four bytes for each block of a float data type and 16 bytes of data for each block of double data type and so on each block it selects will have a memory address along with it and a space for data to be stored once the data is stored then the last memory block will replace the address with a null value which indicates the end of the link list using the interconnection between the notes the link list behaves like an array and displays all the data in the form of a sequence now that we have understood what exactly a link list is let us discuss about the types of Link lists the link lists are majorly divided into three types the singly link list the doubly link list and the circular link list a singly link list is a standard link list which mainly consist of two Fields the address space to store the address of the next node and the blog for storing the data as you can see this particular node is the head node and this particular node is the tail node this is the block which stores the address of the upcoming node and this block stores the data we can Traverse in one single Direction in a singly link list the next one is the W link list the W link list is completely similar to a singly link list but the only difference is that the nodes in the W link List have three Fields namely the address note for the next upcoming note the memory block to store the data and finally the block which stores the address of the previous node as you can see this particular Noe is the head node this particular note is the tail Noe and these notes are the intermediate notes which are lying in between the head node and the tail node as you can see this particular block of this node is pointing to its previous node and this particular block of this node is pointing to the upcoming node and this particular block is designed to store the data which you provide into it this makes the dou link list Traverse in both the directions now let us move into circular link list the circular link list is a singly link list but the only difference is that the final note address block is not ended with a null value instead it points to the address of the head node as you can see in this particular link list the address of the first block is pointing to the next one and the next to the next one followed by which we will reach the tail node in the tail node the address here is not null instead it is pointing to the address of the head node that’s what it makes as a circular link list now let us discuss some important features of Link list the important features of Link list are first one is q and DQ interface the link list implements q and DQ interface therefore it can also be used as a q DQ or a stack the next one is it can contain all the elements including duplicates and null the the thirdd one is link list maintain insertion order of elements the next one is Java link list is not synchronized that means in a multi3 environment you must synchronize concurrent modifications to the link list externally we can use collections. synchronized list to get synchronized link list link list class does not Implement Random Access interface so we can access elements in a sequential order only we can use list iterator to iterate the elements through the list now that we have understood the features of the link list let us move into the methods that we can use in the link list there are more than 40 methods that we can apply in link list but here to save time let us discuss the crucial methods required to actually work with link list here as you can see I’ve used the link list method which is provided in Java and I’m creating a link list and then I’m trying to add the string type values into my link list using the link. add method and find finally we shall also use collection method which is previously described in Java to add few more elements which are of string type into our link list and try to append them in the tail section of the link list let us try to execute this program and see how does it work as you can see the output is been printed here the first data which we tried to insert into our link list was welcome to edureka and after that we tried to append the data of string type into the end part of our link list which happens to be the tail and the link list after appending the string data type at the last is welcome to Eda an online technology training center now let us try to insert the data at the head part of the link list so in this particular program I’m making use of the link list method which is previously described in Java and I’m adding the elements welcome and Eda which are string data type into the link list and in the next section I’m using collection method and I’m adding few more elements into the link list at the head section of the link list now let us try to execute this program and see how does it work as you can see the output has been successfully generated and the data of the link list before appending the data to the Head section was welcome urea and after appending the data to the Head section we have welcome to the most popular online technology training center edureka now let us execute add method in this particular method we are just appending new elements into the link list similar to the previous methods I have created a link list using the link list method which is previously programmed in Java and I’m trying to add up the elements into it as you can see in in this first section I’m having few elements called hello world message from Eda and after that in the second section I’m trying to add two new elements which are new and element let us try to execute this program and see how does it work as you can see the output has been successfully generated and the link list before appending the new element to it is Hello World message from urea and after appending the new element we we have Hello World message from edureka and the new elements which are new and element now let us try to append the data using add method by using the index followed in the link list as you can see in this particular example I’m trying to add two new elements which are U and welcome at the index locations 2 and four respectively now let us try to execute this program and see how does it work as you can see the output has been successfully generated and the output before adding two new elements are how are question mark to edura here I’m adding you and welcome into this particular link list in a designated location which is 2 and four so after adding two new elements this particular link list is making a sensible statement which is how are you question mark and welcome to Atura now let us try to execute clear method as you can see in this particular example I have created a link list and I have also added some elements into the link list which is edura online training institute after that I’ll be clearing the link list using the command list. clear once after I clear the list the link list becomes completely empty after that I’ll be using the add method again to add up new elements into the link list so now here the link list will be completely erased in the first half and after that once if I try to insert new elements into it then it will take up the new elements and add them into the previously existing blocks without having the previous data in the link list now let us try to execute this program and see how does it work and understand it in in a better way as you can see the data has been successfully generated and the data which was present in the link list before clearing was edura online training institute once after the data has been cleared successfully the new elements are been added into the link list which are now tied up with nit War Angle now let us try to execute clone method in this particular example I’ll be using clone method the Clone method is nothing but you cloning the existing link list now let us try to execute this program and understand this in a better way as you can see the first link list is example for clone method I’m using clone method and I’m trying to duplicate the first link list which has the same data as you can see which is example for clone method now let us try to execute index method as you can see in this particular example I’m using index of method so here I have inserted some data into my link list which is using index off method so what I’ll do here is I’ll be using list. index of method to find out the index of a particular element so here I’ve chosen using element and Method element to find out the index of those particular elements let us try to execute this program and see how does it work as you can see the output has been successfully generated and the link list was using index of method now the index of the element called using is zero and the index of the element method is index 3 now let us execute offer method as you can see in this particular method I have created a link list and added the elements BMW Posh and Mercedes and I’m using offer last method to add the final element which is Jaguar to the link list so let us try to execute this program and see how does it work as you can see the output has been successfully generated and the elements in the link list before using offer last where BMW POS and Mercedes and the elements after executing the offer last method is BMW Posh Mercedes and jaguar where the element Jaguar is included into the link list at the last section which happens to be the tail now let us execute offer first method in this particular example I’m creating a link list which will be having the elements C language Java and cin and finally I’ll be using the offer first method where I’ll be including the python element in the head section of the link list now let us execute this program and see how does it work as you can see the python element is included here at the head section now now let us try to execute pop method in this particular example I have pushed in two elements which is a and b and here I’ll be using the pop method to pop the last element which is pushed into the stack and after that I’ll try to push a new element which is C into the stack and finally we’ll print the stack which is existing right now let us try to execute this program as you can see the last element which was pushed into the stack was B and after popping it we have the elements C and A which are existing in the current stack now let us try to execute the push example as you can see this particular example is similar to the previous one here we are just pushing the elements into the stack using the stack push method as you can see the output has been successfully generated and the elements which we have pushed into the stack are 100 200 and 300 now let us try to execute a remove method as you can see in this particular example I’m trying to insert four elements into the list which are apple banana and grape and here I’m using list. remove method to remove one element from the link list which is the first element which happens to be apple now let us try to execute this program and see how does it work as you can see the elements before removing the element Apple are apple banana grape and pineapple so once once after I remove the first element the remaining elements are banana grape and pineapple now let us try to execute the set method in this particular example I’ll be using list. set method here I’m trying to set two elements into the link list which are kiwi and orange at the position two and three as you can see this is the element kiwi and this is the element orange and I’m trying to set Ki into the position number two and orange into the position number three let us try to execute this program to understand this in a better way as you can see the output has been successfully generated and the elements before inserting them into the second and third position are apple banana grape and pineapple the objects grape and pineapple which were at the positions 2 and three are replaced with kiwi and orange now let us try to understand size method size method is not a complicated Method All it does is shows the number of elements which are present in an array now let us try to execute this program and find out the size of the particular link list as you can see I’m using list. size over here and I’ll be finding out the size of this particular link list now let us try to execute this program as you can see the link list is apple banana grape and pineapple and the size of this particular link list is four now let us try to convert an array into the link list as you can see in this particular example I’m trying to add the array which is courses which has Java PHP Hardo devops and python elements into a link list so here I’m using for Loop into which I’ll be adding the string s Elements which are courses into the link list which is a course list using the add method now let us try to execute this program and see how does it work

    as you can see the elements Java PHP Hadoop devops and python were actually the elements of an array called courses and using the add method we have added the elements of an array into the course list which happens to be the link list and here the output has been printed successfully which happens to be the output of a link list now let us try to convert a link list into an array in this particular example I’ve used add method to add the Java python devops Hardo and AWS elements into a link list called as course list now we’ll be using to array method to add all the elements which are present in a link list into an array now let us try to execute this program and understand this in a better way as you can see the size of the link list was five the elements of the link list were Java python Hadoop devops and AWS which are now included into the array called as numbers which which is displayed as follows Java Python devops hadu and AWS so these are the elements which are present in the array called as numbers so we have successfully converted a link list into an array now let us discuss the major differences between arrays and Link lists the arrays are not resizable because the size of the array has been defined at the Declaration section itself but on the other hand link lists are dynamically resizable the next difference is the ARs offer more methods compared to link list while the link list are concise to few methods compared to arrays the third difference is the arrays have insertion addition and removal operations done in a faster way while in link list you have to Traverse from head to tail the random axis is not provided by link list so these operations are comparatively slow in link lists the last difference is the AR consume more memory because hardly the aray memory which you allocate is been reused but in terms of Link list the memory which you use can be dumped back into the Heap and it is readily available for reuse so Aries consume more memory and the linkless consume less memory let’s understand what is a hashmap in Java hashmap is a part of Java’s collection since Java version 1.2 it provides the basic implementation of map interface in Java it stores the data in key value pairs to access a value one must know its key hash map is known as hashmap because it uses a technique called hashing so what is hashing it is a technique of converting a large string to a small string that represents the same string a shorter value helps in indexing and faster searches hash set also uses hash map internally it internally uses a link list to store key value pairs so the important features of hashmap are it is a part of java util package it extends an abstract class that is abstract map which also provides an incomplete implementation of map interface hashmap also implements clonable and serializable interface it does not allow duplicate keys but it allows duplicate values that means a single key cannot contain more than one value but more than one key can contain a single value hashmap allows null key also but only once and multiple null values and this class makes no guarantees as to the order of the map in particular it does not guarantee that the order will remain constant over time it is roughly similar to hashtable but is unsynchronized so these are the various features that makes it the popularly used data structure next talking about the internal structure of hashmap it contains an array of node and node is represented as a class which contains four fields that is inh K key V value and node next next it can be seen that node is containing a reference of its own object so that is the reason it is called a link list so very simple it compris of a node which compris of a key and value and it has int hash and node is having the reference of the next variable as well so this is all about the internal structure of hashmap now talking about the performance of hash map it mainly depends on two parameters that is initial capacity and the load Factor so capacity is the number of buckets where the initial capacity is the capacity of the hashmap instance when it is created the load factor is a measure that when rehashing should be done and rehashing is a process of increasing the capacity in hashmap capacity is multiplied by two and load factor is also a measure that what fraction of the hash map is allowed to fill before rehashing ing when the number of entries in the hash map increases the product of current capacity and the load factor is also increased that is it implies rehashing is done if the initial capacity is kept higher then rehashing will never be done but by keeping it higher it increases the time complexity of iteration so it should be chosen very cleverly to increase the performance the expected number of Valu should be taken into account to set the initial capacity most generally preferred load Factor value is 75 which provides a good deal between the time and space cost and the load factors value varies between 0o and one based on these two factors that is initial capacity and a load Factor performance of a hash map is measured now let’s move into the next section and understand what is a synchronized hashmap as it is already told that hashmap is unsynchronized that is multiple threats can access it simultaneously if multiple threads access this class simultaneously and at least one thread manipulates it structurally then it is necessary to make it synchronized externally it is done by synchronizing some object which encapsulates the map if no such object exist then it can be wrapped around collection synchronized map to make hash map synchronized and avoid accidental unsynchronized access so you can see that I have used collections. synchronized map and created a new hash map so by using this a map m is synchronized iterators of this class are fail fast if any structure modification is done after the creation of iterator in any way except through the iterators remove method in a failure of a iterator it will throw concurrent modification exception so this is how you can make a map as synchronized now let’s see some of the constructors that are widely being used hashmap provides four Constructors and access modifiers of each one of them is public first one hashmap it is a default Constructor which creates an instance of a hash map with initial capacity 16 and load Factor 75 next hash map with in initial capacity this also creates a hashmap instance with specified initial capacity and a load Factor 75 next hashmap with in initial capacity and Float load Factor it creates a hashmap instance with specified initial capacity and a specified load Factor next one that is a last Constructor hash map which has arguments of type map it creates instance of hash map with same mappings as a specified map so now let’s write a small program and understand how does do the hashmap work so I have created a class called hashmap and inside that I have a main method and you can see I’m using hash map and I’m passing the parameters like string and integer and I’m mapping it with the object of a new hash map so first when I print the map it should say this map is empty because nothing is there in the map right so then I’m using the put method to insert the elements into the map that is ABC 10 m 30 and XY 20 then the when I print this it will say the size of the map that is three then next I will check whether a map contains a key called ABC so this is the keys and this is the value of the respective key right so when I check if it has a key called ABC then I’m using a map dog to display the value of ABC so the value for key ABC will be whatever is present in this right then again I’m clearing the map and again printing the map now I’m using a method called print I’m passing the arguments for map means it should contain the map of same mappings so if it is empty then print map is empty else return the elements and that is the key value pairs present in the map so let’s run and check the output so first what it said it said map is empty because I’m PR printing the map without entering anything in that so it will go to the print method and check whether the map is empty or not yes it is empty so it will retrieve the result so next this statement got executed and it’s telling the size of the map his three because it has the key value for three different categories again it will print the map so this will again go to the function definition and print this statement because now the map is not empty so there’s a function call over here and now size of the map is map do size what is the size three so it is displaying the size of the map as three and is printing all these elements now value for key ABC is 10 so that’s what I’m checking over here that is get the key for ABC and print its value so if I do it again like you know m n o again change it over here as well and and here as well okay so it will print the value will be 30 so finally after clearing the map again it’s telling map is empty now let’s run and check the output it’s MN o you can see that the value for the key m o is 30 right I hope you understand this so this is how you can configure your hashmap and map it accordingly so now talking about the complexity of hashmap it provides a constant time complexity for basic operations that is get and put if hash function is properly written and it disperses the elements properly among the buckets iteration of hashmap depends on the capacity of hashmap and the number of key value pairs basically it is directly proportional to the capacity Plus site and capacity is the number of buckets in the hashmap so it is not a good idea to keep a higher number of buckets in hashmap initi so this is all about the time complexity now let’s have a look at the various methods that are widely being used in a hash map first void clear this is used to remove all the mappings from a map next Boolean contains key that is parameter will be object key this is used to return true if for a specified key key mapping is present in the map and again Boolean contains value of object value this is used to return true if one or more key is mapped to a specified value next method is object clone this is used to return a shallow copy of the object of the mentioned hashmap next Boolean is empty used to check whether the map is empty returns to if it is empty next object get object key this is used to retrieve or fetch the value mapped by a particular key set key set this is used to return a set views of the key in size used to return the size of the map next object put parameters will be key and value this is used to return a particular mapping of key value pair into a map next put all of map M this is used to copy all of the elements from one map to other next object remove object key this is used to remove the values for any particular key in the map and the last one is collection values this is used to return a collection view of the values in the hashmap so these are the various methods that are widely being used in the hashmap let’s understand what is generics in Java generic is a term that denotes a set of language features that are related to the definition and the use of generic types and methods Java generic methods differ from regular data types and methods so before generics we use the collections to store any type of objects it can be either a generic or non-generic object now generics Force the Java programmer to store a specific types of objects so basically this is all about what is gener now let’s move further and see why do you need Java generics if you look at the Java Collections framework then you will observe that most of the classes take parameter or argument of type object so basically what happens in this form they can take any Java type as argument and return the same object or argument it can be either a homogeneous or heterogeneous that is not of a similar type so sometimes in the Java application the data type of the input is not fixed the input can be an integer a float or a Java string in order to assign the input to the variable of the right data type prior checks had to be conducted in the traditional approach after taking the input the data type of the input was checked and then it was assigned to the variable of the right data type when this logic was used the length of the code and execution time was increased So to avoid this genics were introduced so on using generic the parameters in the code is checked at compile time automatically and it sets the data type by default so this is where you need the concept of generics in Java as the execution time and the time that you invest in writing the code will also be decreased so now that you know what is gener in Java and why do you need it let’s move further and see some of the types of gener so basically there are four types first generic type class and you have interface method and Constructor first let’s understand what is generic type class a class is set to be generic if it declares one or more types of variables so these variable types are known as the parameters of the Java class let’s understand this with the help of an example so here you can see I have created a class with one property X and the type of the property is object correct so what happens once you initialize the class with a certain type the class should be used with that particular type only for example if you want one instance of a class to hold the value of type string then programmer should set and get only string type since I have declared the property type to object there is no way to enforce this restriction a programmer can set any object and expect any return value from the get method since all Java types are subtypes of object class so it’s very simple right so here I’m creating a method for set and get and I am using the this keyword for reference and then when I return the value of the X it can be any type but all the Java types are subtypes of object class so I can set a property for integer and can return string anything it can be so this is how it worked the next type is generic type interface an interface in Java refers to the abstract data types they are allow Java Collections to be manipulated independently from the details of the representation also they form a hierarchy in objectoriented programming languages so let’s take an example and understand this so here I have created a interface for gener and I’m doing two operations for T1 and T2 so for T2 what I’m doing I’m performing the execution I’m passing two variables that is T1 and X and next what I am doing I’m performing a reverse execution of whatever I have performed in the above statement so when I create a class for generic and implement the interface whatever I perform the execution will be reverse in the next execution suppose say while performing the execution and passing a string variable and that can be easily converted to a integer variable when I’m reversing the execution so basically all that matters is the type of the object and there is no way that I have to enforce a restriction or anything like that it can be a string integer double float anything so I hope hope he was able to understand how genri can be applied to interfaces now let’s see what’s next the next type is generic type method so generic methods are much similar to generic classes but they differ from each other in only one aspect that is the scope or type of information is available inside the method only generic methods introduce their own type parameters so let’s take an example and understand this so in this case what happens if you pass a list of string to search in the method it will work fine but if you try to find a number in the list of a string it will give compile time error yes right because see first I’m using Advanced for Loop I’m trying to check the element present in the list if the list is an string then it will match because always list is a string like list can be a string it can’t be a integer so when I try to find a number in this list it will give error right so that’s how I have coded this program and that’s how the generic type method tells you that is they introduce their own type parameters there is no way that I have to define something or user have to define something there’s nothing like that it’s like they Define their own type of parameters so this analogy is similar for Constructor as well so if I talk about the generic type Constructors the same goes here as well so basically a Constructor is a block of code that initializes the newly created object a Constructor resembles an instance method in Java but it is not a method as it does not have a return type the Constructor has the same name as the class and looks like how I have written in the code so here in this example Dimension class Constructor has a type information as you can see that so basically you can have an instance of Dimension with all the attributes of a single type only for example you have a class Dimension that is of a type T and I have created three variables when I create a Constructor for all these three values you will get the length width and a height by using the this reference because I’m giving this this do length is equal to length this do width is equal to width and this do height is equal to height so you cannot get any other parameters apart from this in this regard that is you can have an instance of Dimension with all attributes of a single type there won’t be any different types over here so this is how you can use type Constructors so now let’s see a small example of generic and understand how actually it works so here I have created a class called test one and I have passed two Vari that is T and U T is a type of an object o1 and U is a type of an object O2 o1 and O2 are object of type T and U correct so now for this I’m creating a Constructor and then I’m giving this reference that is this do object 1 is equal to o1 and this do object two that is O2 is equal to O2 and now I want to print the objects of T and U so I’m basically creating a method called print and then I’m passing these two variables in this and after that I’m creating a class called generic and inside the main method for the test one for object T and U one will be the string that I will be passing and the other one will be integer so you can see here that I have not created any specific integer or string variable instead I’m just passing the type of an object that is T and U to be a string and an integer correct there’s no restriction or I don’t have to enforce anything over here so I’m going to create a new object for that that is test even and pass the variables as adua and integer value will be 10 so when I give object do print this object is nothing but this reference it will call the method or the object from this print so what will happen it will display the elements present in the o1 and O2 so the elements present in o1 and O2 is nothing but the edua and 10 so when I execute this let’s see what happens I’ll run this as a Java application and let’s see so you can see here that in the output screen it set edura and 10 which means there is no type restriction over here it’s very simple as you saw all that refers to is a type of the object like all the Java types are subtypes of object class and that is the reason I’ve taken object class and referred all the elements as objects and then pass whatever I wish to that is a string and an integer and then I print it so this is how it works now let’s see what is generic functions so we can also write generic functions that can be called with different types of arguments based on the type of the arguments passed to the generic method and further compiler handles each of these methods so let’s see how it works so I have a class called test in this I have created a method that is generic display and pass the element as St element okay so now I’ve have made this as static and I’m using the generic type t Okay so next when I’m giving system.out.print Ln element. getet class. getet name plus element okay so you can see here that in one single print statement I’m trying to get the class get the name and also the element so let’s see how this one single line of code works so inside the main method I’m calling generic method with in argument okay so generic display is nothing but this method right so the T element that is the type of element is integer so it is 3 4 5 6 correct so next I’m again calling the generic method with string argument that is I’m calling the method that I’ve created above and I’m passing the element of type t as a string and after that we’re double so you can see here that there are three different types of elements that is integer string and double and all these are of different data types right so when I say get class. get name plus element it will say the class is this and the name will be the integer type and the element will be the value yes let’s execute and then you will understand what actually that statement does as I told you element. getet class. get name is equal to element right so element is Java do Lang is a type of the class do integer is a type of the name first one is integer so element. getet class. get name is equal to this value so same goes for next also element that is Java do Lang is the get class do string is get name and Eda is a value same goes with double as well so you can see here multiple types of arguments was being passed at one single instance so this is how the generic function works as you can pass n number of arguments by just referring to it as a type T there is no specification there is no restriction that you have to enfor it on all that you have to do is create a object of a type generic and then go on adding the values simple yes that’s how generic works now let’s move into the last part of the session and understand some of the advantages of genri and Java first code reusability you can compose a strategy or a class or an interface once and use for any type or any way that you need next type safety as you just saw that whenever you use different types of arguments and when you casted in different types of arguments it was very safe right there was no hampering of the code or there was no destruction of the code or there was no any type issues when you try to cast the code or when you use various types of code there was no hampering between those codes so obviously it this type safety individual typ casting is not required basically whenever you recover information from array list every time you need to typ cast it but T casting at each recovery task is a major migraine so in order to eradicate that approach genics were introduced as you just saw you can use various data types and there is no casting of that required because you can use all of them at once and all that refers to as an object so that’s why I think type casting is not required next implementing a non-generic algorithm it can calculate the algorithm that works on various sort of items that are type safe as well it can be a generic or non- generate no matter what it will Implement everything let’s understand what is file handling in Java so file handling implies how to read from a file and how to write to a file in Java and Java basically provides the basic input output package for reading and writing streams and also java. inputoutput package allows you to do all the input and output tasks in Java so in order to use a file class you need to create an object of the class and specify the file name or directory name as shown below so first you will write import java.io file that is used to import the file class and then you have to create the object of a file and specify the file name simple so let me tell tell you one thing Java uses the concept of stream to make input and output operations on a file so let’s Now understand what is stream in Java so the stream is a sequence of data it can be of two types bite stream and character stream now talking about the bite stream it mainly incorporates with bite data when an input and output process happens with a bite data then it is called the file handling process with bite stream now talking about the character stream it it is a stream which incorporates with characters when an input and output process happens with a character then it is called as the file handling process with bite stream so these are the two types of streams that are available now let’s move further and have a look at the various file methods that are useful to perform Java file operations first can read this method is used to test whether the file is readable or not next can write this method test whether the file is writable or not and next you have a create new file which creates an empty file and next delete this command is used to delete the file exist this test whether the file exist or not and next you have get name which is used to return the name of the file and you have get absolute path and this is used to return the absolute path name of the file and next you have length and it Returns the size of the file in bytes now moving further you you have list this method is used to return the array of the file in a directory and you have mkdir which is used to create a directory okay now these methods are used to perform various file operations now let’s see what are the various file operations that are present in Java so first you need to create a file then once you create a file you need to get the information out of the file after that you have to write the information to the file and then you have to read the data from the file correct so now let’s understand all these things in a much better way with the help of an example first create a file in this case in order to create a file you can use a create new file method okay and this method returns true if the file was successfully created and it returns false if the file already exist okay so here is an example for creating a file so first I have created a package and inside that I have a class called create file as I have already told you I need to import the file class so that is the reason I’m using this package okay and next in order to import the input output exception class to handle the errors I have java. input output. iio exception okay so next after that I have created a class called create file and inside the main method I’m using two blocks that is try and catch so inside the triy block what I’ll do I’ll write a code that has to execute and the catch block so if there is any error that is going to occur in Tri block it will be handled and in this case most expected error can be the io exception so that is the reason I’m using a catch block to handle that okay so inside the tri block let’s see what I have written so I have written file my object is equal to I’m creating a new file in the specified directory if I don’t give the path if I just give the name of the file that is file f.txt it will by default go go and store this file in the location where my Eclipse workspace is so I don’t want to do that I want to create a file in a specified file location and that’s a reason I’m giving this and now if my obj that is my object do create new file then if the file is being created with this object then it is telling file created and get the object name that is get name so get name will be what file F not txt right else if the file already exist it will say file already exists and if there is any exception it will be handled in the catch block so now let’s run and check the output so you can see that it’s telling file created and it returned the name of the file that is my obj doget name that is file fn. txt correct so yes that’s how it works now let’s cross verify this I’ll go to my D drive I will go to file handling and you can see and you can see file f created correct so in this location itself I have created the file so if I run this program one more times let’s see what happens so it’s telling file already exist because one time you created a file it will be present in that particular location so it will say file already exist cor right so this is how you need to create a file so the next step is to get the file information so here again I’m importing the file class and then I’m creating a class called file information and inside the main method I’m creating a new file okay and I will check if this already exists then it should print the file name get file name that is my obj doget name and then it should again print the absolute path of the file and for that I’m using get absolute path method and then I’m using if it is writable it will print can write if the file is readable it will say can read and the file size and bytes will be my obj do length Okay else if the file does not exist then it will print file does not exist so you can see that I have used all the methods that I have explained you before that is get name get absolute path can read write and length right so let’s run the program and check for the output so you can see that the file name is file 1.txt the absolute path is it’s in this location that I have already showed you just now for writable and readable it’s telling true because yeah it is readable and writable and the file size in bytes is zero so I have one more file here let me show you that I have a file called new file 1. text okay so inside that I have written some text so it means it consists of a text right so when I say the file size in bite you has to return the length but here it is telling zero because I have not written anything into that file so now if I change the file name let’s see what it will return for the statement let me run and check it once again so you can see that it return the file name absolute path it is readable and writable and the file size in bytes is 52 correct so if you have written anything in the file then it will on the file size or else it will not simple so this is how you need to get the file information next let’s see how to write to a file so here I have created a class called Write to file and inside the main method again I have two blogs that is try and catch and here I’m using the my writer method so here I have used the file writer class together with its right method to write some text into the file let’s Now understand how actually works so I have used a file writer and I’ve created a object of the writer and this is a path where I have saved my file and this is where I want to write some data into the file right then I’m using my writer’s write method to write this particular context in the file and after that I have to close it so once I close this I will say successfully rot to the file okay if the data has been written into the file then it will say I have successfully written the data into the file else it will throw an error and will be handled in the catch block let’s now run and check the output so it’s turning successfully wrot to the file now let’s cross check this so you can see that it has written the data into the file that is Java is a prominent programming language of the Millennium if you want to do any editing over here you can do which means is writable and if I want to read the data from the file I can read it as well right correct see so simple it will even ask whether to save or not if I say save yes it will get saved very simple right so that’s how you need to write a data to the file now that you have written the data to the file you know that you can go to the particular file location and check whether the data is been written into the file or not but how does Java know that right so now let’s write a Java program to read the data that you have written into the file again I’ll create a class called read from file and inside the main method I will write file my opj is equal to new file and this is the path of my file and then I’m using scanner and my reader by because I want to read it that is a reason and I’ll pass the parameter as my obj my obj refers to the file which is this one right and next while my reader do has next line so if it has next line or if there is n number of data that is being present in that it will read everything that is the reason I want to check whether it has a next line ifs again string data is equal to my reader. next line and it will print the data and after everything it says my reader. close because I have to close this and finally if the file has not found then it will throw an error and will be handled in the catch block simple now let’s run and check the output so you can see that it return Java is a prominent programming language of the Millennium which I had written from this thing right so from the right method and this I manually type so it will again print the data that has been present so this is how you need to read the data from the file I hope you understood how to create a file how to get the information from the file how to write to a file and how to read from a file so these are the various file operations that you can perform on a file so that’s how it works and that was all about the file operations what is a Java thread so guys a Java thread is a lightweight subprocess now in an application so we have one main thread so when our application is uh running so operating system is going to create one process so same way over here Java virtual machine will take care of your Java applications so process proc would be created and each and every process will have a main thread so thread is a lightweight subprocess so here it is the smallest independent unit of a program it contains a separate path of execution and every Java program will have at least one thread that is something known as the main thread so main is nothing it’s an execution context where the certain jobs will be executed one after the other in a sequence now if you want to create a thread in Java so we got this thread API from the java. Lang package itself you need not to even import it so it’s by default in the Lang package once we create a thread it will have a life cycle so how it’s going to work right so a thread can lie only in one of the shown States at any point of time so typically guys we’ll have a thread with a state called new so when you create a thread its state is New then we move into the runnable then into the running and finally the thread will be terminated in between a thread can have the waiting state which can uh iterate between your runnable and running so let us have a look onto uh individual states what exactly is what state so having a new state means that we are going to initiate a new object of a thread you can even say that the thread is just born right it’s just like creating an object of a thread in the memory thereafter once a thread is created we start the thread so once we start the thread the state is changed to the runnable state right so that it can run and finally your thread is now running so running means it is executing a run method which is the method we have to override from the thread class right and we even have the yield method that can send them to go back to the runable state again what is baiting State when a thread enters a state of inactivity so thread is uh not performing any action it is waiting for some other thread to give back the data or it is sleeping or it is blocked at certain scenario or a use case so that is known as a waiting State your thread is not currently resuming the tasks finally the thread is terminated so when your thread has completed its task right so it is terminated now let us take one example of this thread consider that you are going to play one audio file in a media player right so let’s say you have a media player and you’re going to play an audio file now the moment you tap on the button to play that audio in your phone or in your PC right so a new thread is created and you see that fin the thread starts proceeding so your song starts to play so from the runnable it enters into the running State and it keeps on running till the song is not finished right so that’s like the terminated so in between you just go for the pause right that’s like where your thread is now doing a weight operation so when your song is automatically finished or you stop the song that is the terminated state so one of one of the just an analogy that how the thread is going to work for us same way there can be a use case that you want to download some image from the server now threads are typically used when we have to perform a long running operation right so there we create this thread downloading an image can be a long running operation because it is dependent on the network speed now when you want to download an image so you create a new thread so in the background what we will do is we’ll start fetching the bytes from the server for the image right and uh in the running State your thread is continuously fetching the data there can be a state called waiting let’s say network connection has been disconnected so our thread can wait until the network connection is you know reestablished and terminated would be a state when the image is properly downloaded so couple of examples demonstrating how life cycle of the thread is supposed to be so now we will look into how we can create a thread so guys if you want to create create a thread right you need to understand the reason why you want to create a thread so we by default will have a main thread in our application that is represented by the main method so what is a main thread right so typically whenever you run your Java application or any application you will write your code in the main method now the code in the main method is executed as a main thread itself right so main method will have the instructions written within it and all those instructions are being executed as a part of the main thread in a sequence one after the other now what will happen if you are going to do a long running operation within the main itself it can take time right so the below written instructions in the main method they will be blocked because of a long running job right now that is where your operating system it will start giving messages to the user that this application is not responding would you like to wait or kill this application so this is what we need to understand right so when we have long running operations we need not to put the load on the main method or the main thread we need to create a separate thread so that is what we going to understand now so creating a thread is like what now we have two different ways by which we can create the thread either we extend from the thread or we can implement the runnable interface right so guys having a thread class so we’ll we’ll just say extends thread any class name extends thread you overwrite the run method you create the object of your thread class and we invoke the start method which will internally execute the run method same way so here you can see a bit of code snippet coming in so there can be a class called my thread extending the thread we have a run method which will do some job and in the main method you create the object of the thread and you just say start so start method will internally execute the run method the way we got thread class in a similar way we do have something known as runnable now many of times uh we are already into inheritance right so let’s say your class is inheriting some other class now it cannot inherit thread at the same time because multiple inheritance is not supported in Java so what we got we got runable interface so if you are already extending some class you cannot extend the thread class but we got a runnable interface which can be implemented rest of the structure Remains the Same right so you create you override the run method create the object of the class and invoke the start method so guys here we go right so your thread class will Implement runnable so let’s now do a little bit of coding here and understand that how it’s going to work for us right so I’m going to write a new Java project we’ll see how we can create threads using uh the thread API and the rable interface so I’m going to see threads demo let’s see next and say finish so we got a small Java project created here in the SRC I’m going to write a new class and I’m going to say threads with the main method or I will say this as app with the main method right so app means application Here app with the main method now when we have this uh main method coming in so guys uh we will come here and say that main method represents main thread right so main method represents main thread so whatever we write in here will be executed by main thread and threads always execute the jobs in a sequence so whatever the job you going to give to the thread they’re going to execute in a sequence for example I will write down one of the job it goes like application started and thereafter I can have a job called application finished so in between I can have let’s say a small job it will print some documents for me right so I’m going to say that for INT of let’s say document one till the document number 10 and document plus plus so I’m printing some documents right so let’s say this will be some code snippet right so some code to print the documents right so this guy is printing some documents ments for us so I will write down the ciso statement here so printing Document Plus the doc right so whatever the document we are printing so I’m just putting up here now when I will run this program it is all ceso statements it’s nothing complicated to understand I hope so so your application started we printed the document one till the document number 10 so we printing these 10 documents in a loop so you see everything is happening in a sequence so what I’m trying to say is that this goes like your job one consider this entire snippet as job two and this goes like let’s say some job three so we are not getting job three before the job two or job one after the job three so what is the output output is a sequential output right now that is what a thread is so what is a thread thread is an execution context so what does this mean this means that it’s going to execute your code in a sequence threads always execute code in a sequence right so one after the other so this is one of the demonstration regarding the main thread which is nothing represented by the main method now what I will do is I will write one class here now this class goes like some my task which extends not extends let’s say my task so my task is going to execute a task for me so this will execute a task for me here what I’m going to do is I’m going to say C so so even we’ll put up a loop here and this Loop goes like a similar Loop so printing the document number document right so here I’m going to put some different D limiter so that we can understand printing document number this from printer one right so this is from let’s say printer 2 and coming here I’m going to say this is your printer one so this is how we going to come up and understand so my task over here is having a method which is going to print the document but this guy will print the document from the printer tool and this guy over here is going to print the documents from the printer one so within the job one and job two I’m going to introduce this job called my task so my task is a new my task right and coming here I will say task. execute the task if you will see now what we have done we have made my task as job two and this guy has job three and this goes like job four now when I will run this code what you will find is that application started printing of documents started so printer 2 is in use and thereafter printer one is in use right so as per our discussion main is executing everything in a sequence and to your surprise you need to understand this point here that till this task is not completed below written jobs they are blocked I need to now mention this point till job two is not finished below written jobs are waiting and are not executed so this is something which we need to understand because of sequential operation here right so we are we are seeing a sequential operation so that is the reason till job two is not completed below mentioned instructions or the jobs they are waiting now in case job two is a long running operation right so in case job two is a long running operation consider this guy is going to print several documents right there can be some several documents which we are supposed to print now till these several documents are not printed right in case job to is a long running operation that is several documents are supposed to be printed right so this is a use case which so in such a use case os/ jvm shall give a message that application is not responding right or we will have some you can say Behavior some sluggish behavior in the apps now what is meant by this sluggish behavior in the app it means means that we will be able to see some kind of apps which are hanging right so you see that apps they are going to hang right so even I can say apps app seems to hang now why app will seem to hang because your main thread is blocked right so the only Point here is that your apps they might seem to hang they will have some sluggish behavior in case you are doing a long running operation within the main thread now this is the use case where we need a thread so I think the use case is clear right so what I’m going to do is so this my task you come up with this my task in the same way so let me comment down this code part so instead of having it a normal class you make it a thread you overwrite this run method with the public because run in the parent is public so you need to override it so this guy is is override it here for us right the run method so I get the same operation being performed but this goes in the run method and my task is now a thread right so I can very simply understand that my task is a threat because inheritance makes this relationship to come up as is a relationship right so my task is a trade so guys this is certainly clear to everyone I hope so so in the next part of this is like what now so you won’t say execute the task so rather we will come here and say task dot start so start method internally calls the run method right so start shall internally execute run method now let us reexecute the same code and see what is the outcome so we see a similar output here right but if you can observe application finish is coming somewhere in between previously it used to come in the end so I’ll give a rerun again now I see a different output right application started sometimes printer one is being used sometimes printer 2 is being used and there after lot of times printer two then sometimes printer one and then the app getting finished so again rerun this application you see again a different output so every time when you will execute your code over here what you will see is a different kind of an output right why now the outputs are mixed for us because along with the job three job two is executed parallely right what I can say now so now Main and my task are executing both parallely or con concurrently that is the reason you are able to see some a mixed output for both of these threads so guys when you write a thread thread’s going to work parall to your main thread right so now my task is also known as a child thread or sometimes the terminology can be a worker thread so child thread or overcut thread right so this guy is going to perform certain operation which will not affect the performance of the main method or the main thread right so we are having this main thread which is not supposed to be getting affected by the operation of my task because my task is executing separately main is executing separately within the same process of course so they are not interfering in a way that it has to be waiting for one guy that it should stop it should finish its execution then the main will resume so we are getting the things executed in a parallel operation now the same thread called my task can also be created so considering that you have a class called someca right and I’m just going to commment down this guy now so considering that we have this my task extending CA and thread both now this is not allowed in Java multiple inheritance is not supported right this is an error why because multiple inherit isn’t supported so what we will do so we’ll come here and we will say extend CA but don’t come up with the thread you can say implements runnable so that that’s just one basic change here rest everything Remains the Same so guys in case your class is already a child of some other class right so if you are already having this inheritance implemented so you won’t be able to write comma thread multiple inheritance isn’t supp in Java so you can anytime come up and say implements runnable coming here you are now going to do it in a different way right so let us see how we can create the object so you can say runnable R is a new of my task so you write a polymorphic statement the reference variable of the interface is pointing to the object which implements it then I will create a normal thread object I will say thread task is a new thread within which I’m going to pass this runnable reference and then I will say task. start so this is a bit of change when it comes to runnable versus your threads so you run this code here so it’s going to work in the same way for us right so it’s again a thread so printer one or printer two right so they are they are now the mix outputs mixed outputs they definitely show that our jobs are getting executed parall so main is not getting affected now right so that that is how we are able to understand this point so every time you will run this code you might end up in looking the different outputs why different outputs because execution of threads is not in our hands it is in the hands of jvm jvm can perform time slicing it can give some time to the main threat sometime to the my task right so that’s how we are able to understand and the threads in Java guys I hope you have understood it so now we shall be proceeding towards the next part so the next part over here is to differentiate between thread class and the runnable interface right so guys there is some common things and some dissimilarities so over here in thread and in runnable so a unique object is created for both of the cases right so memory consumption of course increases because we have a separate thread running in the same process then class extending the thread cannot extend any other class because of the multiple inheritance but here runnable helps us to perform this same uh use case so if you are already extending you can Implement a runable so thread class is extended only if there is a need of overriding the other methods of it so if you are actually inheriting the thread class so you might be looking forward to override its other method as well but in case you don’t want to do that runnable is the best use case right because runnable you can only have this run method which you can override then having thread we got tight coupling and runnable is loose coupling so of course guys when you have a loose coupling dependencies they gets reduced and is a better use case to use so moving ahead now so I hope tread versus runnable is is good to go we saw how to create a thread what is a thread and how it is helping the main thread to accomplish the jobs so what is this main thread once again main thread Revisited guys main thread is the important or the most important part of any application or a Java program so whenever you run your application your main thread is executed so every program will have this main thread and Java main thread is needed so that we can spawn the child threads we can create child threads we can start them you you have seen that we created my task we started the my task right so it must be the last thread to finish the execution that is the main thread stops the program terminations so it’s it’s going to terminate the program so when your main thread is finished so it is basically a termination of your application itself right so Java main thread guys how it’s going to work let’s see that part now so typically jvm is going to start the main thread and some other demon threads at the same time right so when you run your application the main thread and the demon threads they are started by the jvm your main thread can further start multiple child threads for example child thread a child thread B and the threads can further start the other threads for example thread a can start the thread C so if we talk about our example here so there can be my task there can be your task for an instance we can have uh some other guy called your task right so this guy is working on some printer three right so we we can have these multiple threads which can be spawned in the main method itself so here I can just say thread your task is a new thread in between I can pass a new of your task and then I can say your task. start right so it’s one the same story either you write this polymorphic statement or you directly pass the reference so these two uh instructions can also be you know written in a way of a new thread and in a new thread you pass a new of your task and then you simply say the start right so you can conclude it in a single statement in case you want if you don’t want to have the reference uh to the thread so these threads right so they can be multiple threads here and there so now we see multiple threads in action right so if you want you can have your threads in a lot of threads executed from the main thread itself even what what we can do is from uh these threads we can execute some other threads that is that is what we were trying to understand and one more thing any thread can be marked as a demon thread so you can just say task. set the demon as true right so what is this demon thread demon thread is a thread which is going to be executed by the jvm whenever the application starts right it’s going to initi it’s going to uh execute the thread along with the main thread if you see the image that is a bit about how the main thread is going to be execute so now next part is multi-threading in Java so typically multi-threading begins when you have more than one thread in your process so of course if along with the main thread a new thread is getting into execution that is a perfect example of multi-threading itself but what we are going to do is we are going to take up some use case that in case there are couple of threads or multiple threads other than the main thread so how they going to work and what can be the uh you know scenarios which we can observe when they are executed so what is multi-threading so guys multi-threading is the ability of a program to execute more than one threads concurrently where each thread is handling a different task at the same time right so that the available resources can be utilized in the most optimal manner so let’s say there’s a main thread and Main thread is going to start some child threads and there can some switching between these child threads right because they are going to perform the jobs and your time slicing will be coming into the action where some jobs of thread a will be finished some jobs of thread B some jobs of thread C right so there will be a switching between these execution of the threads so let us see one demonstration where multiple threads can be introduced so I’m going to write a new Java program so a new class so this goes like my uh let’s say sync app with the main method so this time I’m going to perform a separate uh operation right so consider that here your application started so this is your beginning of the main thread and this will be the ending of the main thread right so that is how we are considering so main is a thread now we understand it right so main is representing main thread all always remember this guys so in order to prove this what I can do is I can uh just try to say in I is 10/ by 0 right and I will run this code here as Java application so what you see is it says there’s an exception and the term which jvm is using is thread main at line number 11 if you see this line number 11 is divide by zero error that’s like arithmetic exception right so what I’m getting is is thread main right so jvm says main method as thread so that is one of the proof of concept what I’m trying to say that why main is representing the main thread right now there is a class called printer so printer is supposed to print the documents so I’ll have this guy called print documents right so printer is supposed to print the documents how many documents you want to print right so in begin let’s say int some number right so I’m just going to take some number here so I will say for INT isign one go till this number and say Plus+ so there there can be even more details in the print document but I’m just taking one of the examples here so I will say ciso printing Document Plus the I so we are going to have this document printing uh done for this IE I can even take one of the name right so I can take doc name and this can go like number of copies what is the name of the document which you want to print and how many number of copies you want so I’ll take this number of copies goes less than equal to number of copies right and printing instead of the document I’m just going to have this document name so I think this is one of the use cases which I wanted yeah now having a printer object so I’m going to write a printer object printer printer is a new printer this is one of the printer object and if I want to print the documents right so what I will do is I will say dear printer please print me the documents I want uh 10 copies for a document called ishan’s profile.pdf so I want I want the 10 documents for ishan’s profile.pdf so when I will run this code here so you see your printer is printing uh the copies so it has clearly said 1 to 10 right now why we should be getting the documents printed here in the main in the main method in the main thread we can have a separate thread right so I’m going to write a separate thread So This Thread goes like uh let’s see my thread which extends thread and I’m going to override this run method always remember guys you can any time have your runnable interface or this thread whatever the way you are comfortable right so here in the my thread I’m going to have this guy called printer PF I’m going to have a reference to the printer in the Constructor so I will take printer P here and I will say PF is p so in the run method here I will say PF do print the document so I need 10 copies so this is like John’s profile or let’s say John’s profile is the name so now so once I created uh the printer object I’ll create the object of my thread so I will say my thread MF is a new my thread and I’m passing the printer over here you see we have one object of printer right how many objects of printer I have created so we have only one single object of printer right only one single object of printer so we are giving the reference so my thread is having reference to the printer object so since my thread is having reference to the printer object so printer is copied to the p p is copied to the p and you are doing a print documents here so I will say MF dot start now what will be the benefit the benefit is that printing is happening in a separate thread that’s like my thread so now when you run this code you see application started application finished so main thread is okay to go with it right and the jobs are getting executed for uh John’s profile by my thread now consider along with the my thread you see you you got this my thread I’m going to write your thread which is very much same all right it’s it’s it’s exactly the same story so instead of John’s profile this is fana’s profile which we want to print right so uh the way we got my thread in a similar way I’m going to say your thread let’s say VF is a new of your thread so it’s going to work on the same printer so you see my thread is having a reference to the printer object your thread is also having reference to the printer object but guys what is the use case or the scenario scenario is that we have multiple threads working on the same printer object now this is a very difficult use case so you see printer is now going to be shared between two threads if I start both the thre threads now so I say yf. start as well right so you see what’s going to happen so we are getting the documents printed so you will run this code so any time of any point of time what you will see is that the outputs they going to get mixed up for us right so let me just keep on running this part here so you see the outputs they are getting mixed up now so 1 to n for the John then 1 to 10 and then the John profile so it it’s totally you see the output coming here now right it’s all mixed up can you imagine such a scenario in a real time envirment so let’s say two people they want to print the documents in the printer can you uh imagine printer printing document one from the first fellow and then the document two and then document three then four so it’s all mixed up right so rather what printer will do printer will take the commands from the multiple people multiple computers but it will print one after the other so this is one of the use cases which is not good to go guys right so this is this is certainly not what we want so what is this this is known as asynchronization threads are running parallely now there can be a use case there can be a scenario where multiple threads so if multiple threads are going to to work on the same single object we must so we must synchronize them so they are not they are not supposed to be executed in the sequence so what I’ll do is I’ll introduce one more uh method here in this for Loop here so I will say a thread dot sleep for let’s say 500 what is this thread dot Ste guys Sleep Method will temporarily pause the execution all right so I’m going to select this guy and say surround with tri catch because Sleep Method will throw an exception now when I run this program here so what do you see is along with some delay the output is coming up for us right so let me run this code once again but this output is not at all acceptable since my thread and your thread are working parall on the same object so it is printing the documents sometime s for my thread sometime for your thread which is not a real life use case this is implemented something in a wrong manner right so what I can do is immediately after the start method I can say mf. join so what does this join method is going to do so it will again throw an error so we’ll surround it with the TR catch block so now what you will observe is synchronization coming into the action let me run this code what you see is after putting the join all the documents are printed from the my thread and then it is your threat in execution let me re-execute the program for you people so guys you see your John’s profile till John’s profile or John’s papers are not finished Fiona’s paper were not started that is what we are able to understand as synchronization that is by having this join method coming into the action so if you have many people who are going to you know uh get this printer utilized you cannot put

    join on each and every uh you see method call on on every thread so what I can do is I can come here and I can just mark my method to go as synchronized so what is synchronized method now so when your thread is going to execute this method called print documents it will acquire a lock we call it an intrinsic lock right now once the lock is acquired no other thread can access the print documents so rather than having the join function call what I’m doing is I’m marking My Method as synchronized so if multiple threads are going to access the same method again and again so what we are going to do is we are going to synchronize them then and now when I will run the code what you see is the same output coming in for us right what is the benefit here the benefit is now we need not to put join method calls on each and every thread right so we got this synchronized uh method itself we can come here in our thread we can say a synchronized block instead so synchronized method was acquiring the lock on the method right so no other thread can access it if one thread has acquired the lock so this is acquiring the lock as a block now no thread can execute or access the printer object till this block is not finished right so this synchronized block has to go in your threads so there are two different ways by which we can Implement synchronization using the synchronized keyword so the output will again remain the same for us now where is is synchronization needed in multi-threading we need synchronization in multi-threading when both the threads or multiple threads they are working on the same single object now that is the use case where you need to implement synchronization so guys I hope this is clear to everyone what is the synchronized keyword so now let us come back so guys the way we have synchronization in a similar way we have some something known as a thread pool right so thread pool will manage the pool of worker threads and contains a queue that keep the tasks waiting to get executed so there there will be a Que Now we will have lot of tasks right for the application which we have to do and a pool can be maintained for these threads which will be executed automatically by this API called thread pool so explore this part so what are the various thread math guys so in order to create the threads we got a thread object and then say start right so we can join the thread there’s something known as a thread do sleeve there can even be an interthread communication so guys we have wait and notify as methods and a thread can be marked as a demon thread you just say set demon as to what’s a demon thread it’s going to get executed along with the main right so when the jvm is going to start the app it’s going to start the main as well as the demon threads let’s understand what are regular Expressions a regular expression is a sequence of characters that constructs a search pattern when you search for a data in the text you can use the search pattern in order to describe what you are looking for so for example you have a regular expression like this as shown in the slide and you can see that the cap rep represents the start of the line and a set which is enclosed within the square brackets represents letters numbers underscore hyphen everything it compris of all the alphabets numericals and special characters as well and also if you want to write a regular expression which is of 3 to 15 characters long then you can enclose it within the curly braces as shown here and the dollar symbol implies the end of the line so this is the basic regular expression where where you can understand how you have to start writing a regular expression and how you can include the alphabets the patterns everything within the square brackets and how to represent the length of the characters in flower braces and finally the end of the line right so a regular expression can be a single character or a more complicated pattern it can be used for any type of text search and text replay of operations a regex character can be simple ABC or it can be a combination of simple and special characters like AB into C or it can be example with braces which includes a d star D plus anything so I hope you understood how actually it looks like now with this let’s understand what is Java regex so a Java regx is an API which is used to define a pattern for searching or manipulating ating the strings it is widely used to define the constraint on strings such as password and email validation so you can see in this below GIF over here that it is trying to search for a regx that is a regular expression and it’s trying to figure out whether it is able to match the pattern or not so there are different methods of using Java regex so let’s move ahead and look at the different categories of regular Expressions the first on the list is matcher class this class is used to perform the match operations on a character sequence so this is used to match a character sequence so it compris of various methods the first one is Boolean matches this is used to test whether the given regular expression matches or not next Boolean find this method is used to find the next expression that matches the pattern so once you get the matches the then it will go ahead and find the next expression which will be able to match the pattern or not if not it stops there and then it will again go ahead and find the next one next Boolean find in start so here I’m passing the parameter called inst start and by doing this it will help to search the next expression from the given start number so you have one more method called string group again this is used to return return the math matched sequence so when you get the proper match sequence by using the Boolean mattress and Boolean find then you have to return the match sequence in order to do that you need to write the string group method next in start this method is used to return the starting index of the regular expression and similarly int end is used to return the ending index of the regular expression and you have the group count which is used to return the total number of the math sequence so these are the methods of match class now let’s move ahead and have a look at the various methods of pattern class pattern class is a compiled version of regular expression which is used to define the pattern for a regex engine so again this also compris of various methods first static pattern compile and here I’m passing the parameter as string regex okay this method will come file the given regex and Returns the instance of a pattern I will tell you how exactly all these methods work with the help of an example after this for now let’s understand all the different methods next you have a matcher match which compris of a parameter called car sequence input so this is used to create a matcher that matches the given input with the pattern and the matcher is the one which is the previous one which comprise of a Boolean find buan search in start end and everything and this matcher is used to create a matcher that matches the given input with the pattern so suppose say you have a pattern that comprise of a special characters along with the numericals and alphabets so what it will do it will try to match the given input with the pattern so for that reason you need to use this method next method is static Boolean matches string regex so this works as a combination of compile and match methods next you have string split this is used to split the given string around the matches of a given pattern so after that you have a method called string pattern and this is used to return return the regular expression pattern now let’s see a small example of how exactly it works so here I have an example in this I’m going to check both categories that is match and pattern as well so first what I’ll do I’ll just import the package for regex and then I’m creating a class called regex example and inside the main method I’m giving the pattern as pattern. compile Dot and Dot that is a starting index and the ending index so it implies I can use anything as a starting index and I can use anything as a index but the thing is it should comprise these x and x and between so for that I will use the matcher which is used to match the given pattern right so inside this matcher I’ll write a and b so what it will do it will match the given pattern so this is the pattern and this is the matcher I’m using both the classes that is the matcher and the pattern so I will check whether the given pattern will match the given input string or not and if the string matches the given reg X then it will say match do matches and it will display the output so now let’s run and check for the output okay so it’s telling string matches the given reg X and as I already told you it will always display the Boolean result that is true or false so it is telling true so now even if I give something like this so what do you think will be the output let’s run and check for it so you can see again it’s telling string mat as a regx pattern but then if I remove One X from here and then I write X over here so it does not match the given pattern right so obviously it will say as false so you can see here it’s telling false right so I hope you understood how to write the pattern and how to write the matcher for the given pattern and remember remember one thing it always displays the output in the form of a Boolean that is either true or false so what happened here in this case internally it uses pattern and mature Java regx classes to do the processing but obviously this reduces the code lines and pattern class also contains the mature method that takes the regx and input string as argument and Returns the Boolean result after matching them so this code works fine for matching the input string with a regular expression in Java so that’s exactly how it works now let’s move further and see the next category that is regex character class so again even this comprise of various methods like AB C so it compris of ABC which implies a simple class that compris of a b or c next negation of ABC this comprise of any character except a b or c that is a negation next you have a to z and capital A to Z which impres a range that includes small A to Z and capital letters from A to Z next it has a through D or M through P that is a union that is a through D or M to p and next you have d e or F which is a Intersection For A to Z and next you have a through z except for B and C that is is AD to Z that is a substraction but you have to negate B and C okay and the last one is a through z and not M through P so these are the various categories of a character class let’s again take a small example and understand whether this works fine or not so now let’s check how this works so again inside the main method I’m using a pattern do mat method to check whether the given set of characters matches with the given pattern or not so first it compris of X Y and Z so you can see here all the character set comprise of the simple class that is ABC that is XY Z so when I pass WB c d it will obviously written false because it is not among X or Y or Zed and now if I just pass X it returns to Y because inside this class it comprises of X Y or Zed right and again if I try to pass double x 5 Y and 1 that it returns false because X and Y comes more than once so that’s the reason it will return false now let’s run and check for the output so as I have already told you it returns false true and false so this is the given character class right right and if I try to match the pattern that is out of the character class then it will obviously return false so this is how you can write an example for character class and check whether the given set matches with the input pattern or not now let’s move ahead and understand what are regx quantifiers the quantifi specify the number of occurrences of a character so again even this compris of various methods for example X followed by a question mark it implies X occurs either once or not at all then you have X+ which implies X occurs more than one time and x with a star that is a multiplication symbol implies X occurs zero or more times and if you have a x followed by an N within the curly braces it implies X occurs n times only and if you have a n with a comma then it implies X occurs n or more times and and if you have a parameter like y comma Z then it implies X occurs at least Y times but less than Z times so these are the various methods of regx quantifiers and let’s see even how this works so inside the main method I’m simply printing this thing that is quantifier and now I will try to check whether the pattern matches or not this symbol in the quantifier implies it should occur either one or it should not occur at all right so when I say A Y and Zed followed by a question mark and inside that if I just pass a parameter a it will say true which implies a occurs only one time next in the same category if I write a three times it says false because a comes more than one time so as I’ve already told the symbol implies I either once or not at all again in the same category of class if I write this it will again say false because all these alphabets comes more than one time so that’s the reason again even for this case which says a m n t it is false because m n t is not under this category and a comes twice that is the reason and this should be true because a y set again does a y set as well okay next plus gtif fire implies what it should occur more than one time so a y z plus implies it should occur more than one time so it says true because a comes once even if I write 3 a again it says true because a comes more than one time and even if I write this pattern again it says true because it comes more than one time and even if I write this it says false because mmt is not a matching pattern okay that is M and T are not a matching pattern so this implies it should OCC either zero or more number of times right so when I give this pattern followed by a star which implies and if I write the matcher this it says yes because a y and Zed occur either zero or more number of times so let’s execute and check the output so you can see here the results with respect to the above correct so as I’ve already mentioned you quantifier works with Star Plus and question mark it implies either it should come one time or it should come zero or more than one time or it should occur more than one time so depending on the type of the quantifier you can give your matcher and it will check whether the given pattern matches with the quantifier or not so this is how it works so the last category on the list is regex meta characters so the regular expression meta characters work as short codes so dot implies it can be any character D implies digit capital D implies non-digit s represents any white space and capital s represents non-white space character W can be a word character and caps W can be a non-word character B can represent a bound boundary and capital B represents a nonword boundary so out of all this you can understand that it just represents what a short code right so let’s also check an example of how exactly this works so inside the main method I have written metac characters D so d means digit right so in D if I give ABC it will say false again and in D if I give one it says true why because digit comes only once and in the same day if I give triple four and three it says false because digit comes more than once and inside the D if I give a combination of digit and a character again it says false right so now in case of meta characters capital D so if I give a meta character capital D and if I try to write a alphabet that is a character it says false because capital d means a non-digit but it comes more than once so if I give in the capital D as just one it says false again because it is a digit again the same thing goes with here as well because these are the digits so in this Capital matter character D if I give a match M it is true because it is a non-digit and it comes only once so a meta characters D with the quantifier so if I give D and followed by a star and if I pass ABC then it says true because it is a non-digit and it can occur zero or more number of times so in these cases what happened I just use a single meta character and I tried to pass a string so that was the reason it was telling false now if I give this it obviously says true because it is a non-digit and also it can occur more number of times so here only if I give D+ so it will say true only so this is how it actually works so that was all about the meta characters now let’s move further and see a small demo where I will tell you how to find the given pattern out of the string and also I will explain you how to verify the email validation okay so I have an example over here this is used to find a given regex pattern and it will also check whether the given string match is with our given pattern or not so I have a class called demo and inside the main method I’m using scanner SC is equal to new scanner of system.in I am using a scanner class because I want to read the input from the user so first what I’m saying inside the Y Loop I’m telling enter the regex pattern and I have to obviously use this because I want to scan for the next line if there is anything as such then I’m using a patterns pattern class to compile the given regx pattern that is is the user input pattern and then I will use the matcher class to match whether the given input string will fall under this given matcher or not if not it will say match not found if it will find the pattern with the help of a matcher doind it will say found the text whatever is entered by the user at the starting index of what is the start of the index the matcher group is used to return the Matched Patterns group and the ending index and finally it will retrieve the matcher do end means the ending point so it will say found the text at the group starting at the index the location where it started and the location where it ended if it is not found then it will say matcher is not found so let’s execute and check for the output so it’s asking me to enter the regex pattern so what I have to enter either I have to enter a part of it or I can enter the full thing so I’ll will do one thing I’ll just type YouTube channel right so you can see it is telling found the text YouTube channel the one which I entered starting at index 19 and ending at index 34 so now check 1 2 3 4 5 6 7 8 8 9 10 11 12 13 14 15 16 17 18 19 so it is starting at the index 19 and it ends at index 34 which is this thing right so again if I enter the pattern edura and I’ll keep it as small it says match not found even the cas sensitivity also matters no matter you give the same string again the K sensitivity also matters a lot so if I give welcome it says it is starting at the index zero because this is zero and it ends at the index 7 correct so this is how you can match the pattern with the given input string so this is all about the Searcher and the finder now let’s see a small example of email validator so I have created two classes here one for email validator and the other one is for the testing of a email validator so in this regard I’m performing a email validator test that is unit testing so inside the email validator I need two classes one is a pattern and the other one is a matcher so the email pattern that I’m going to give compris of all this that is this is the starting index as I already told you and this is the ending Index right so it should comprise of all these things plus these things and there should be a add symbol because it’s an an email address followed by again a digits and small and capital letters and again the same thing and it can also comprise of more number of digits and finally it has the ending pattern right so inside the email validator I’ll try to match with the given email pattern and I have a Boolean validate final string hex so this is used to match the hex of the given parameter and finally it Returns the matcher and now I have the email validator testing so here I’m using data provider to provide the data and this will be the unit test case which I will be running as a testng test so if you wish to know more about what is test NG how exactly it works what is unit testing you can check out the videos on our YouTube playlist and you will get Hands-On on even those topics as well so now you have a public class email validator test yeah I’m going to create a object of a new email validator and using the data provider I’m providing the valid email address that compris of a string followed by a add symbol and then it comprise of the domain followed by a particular domain it can be net it can be doom. go anything right which implies it has to comprise of this followed by at if it is Gmail so it has to compile Gmail dot again com com test anything so these are the email address which is valid and I again have a invalid email address so you can see if I simply give us Eda it is not a valid email address right so the email address should comprise of this pattern if not it is a invalid email address again here Eda followed by a do and gmail.com again it is a invalid so again you can see after a gmail.com again it compris of something that is also a nonvalid email address comprise of star percentage so many things right so these are not a valid email address so what I’m doing I’m using ADD test annotation to test the valid email address so here what I’m doing I’m passing the valid email provider so what happens is it will check both the valid and invalid email address and it will display the output so the data provider is used to provide the set of the data and from there you can validate the email address so I’m writing two test one for valid email provider and the other one for invalid email provider so whatever is present in this I’m sending it to a variable Temp and then I’m validating that and finally I’m checking if the result is true then I’m using assert equals to true and if the result is false I’m giving assert is equal to false so let’s check the output so I’m going to run it as a test NG test and let’s check for the output so you can see here first the valid email addresses till here right so these are the email addresses which are valid and that’s true and now talking about the invalid email addresses it starts from here and it is false why because that’s not true that’s reason it is false and it will also return the string for valid email test and the invalid email test so both on the default test and suit the test that was executed was to and there were no failures because whatever I have written the code was working perfectly fine and that’s a reason so even if I have to check the result of running class email validator test you can check all these the tests that was run was two one is valid and one was invalid and this is the class name which is followed by a package on both default suit and test the test was run Within These many seconds yes so this is how you need to validate the given email address and check whether it is invalid or invalid that’s exactly how it works and I hope you got a brief idea about what is regular expression how you write a regular expression what are the different categories of regular expressions and how to configure your program and check whether the given regular expression or a pattern matches with the given input string or not and how to validate a email address as well let me tell you what is socket programming in Java so Java socket programming is used for communication between the applications that is running on different Java runtime environment it can be either connection oriented or connectionless so basically socket and server socket classes are used for connection oriented socket programming I also told you that it also has a connectionless programming and that is achieved with the help of a datagram socket and datagram packet so basically the client in the socket programming must have two information one the IP address of the server and the other one the port number so in order to connect to a server from the client and if the client wants to send a request to the server then it should know the IP address of the network and the port number which the server is operating on so this is all about the brief of the socket programming now let’s move further and understand what is a socket a socket is one endpoint of a two-way communication link between the two programs that is running on the network the socket is bound to be a port number number so that the TCP layer can identify the application that data is destined to be sent so basically a connection will always happen between the server and the client port and obviously when a client sends a request server has to wait for the client request then the server has to accept the request and then establish a connection with the client and processes the request and send the response back to the client so this is what it does and for all these things what you need you need a socket which is an endpoint for a two-way communication between the two networks or the two programs that is running on the same network so I hope you understood this now let’s move further and see some of the methods of socket and socket server classes the first method for socket class is public input stream get input Stream So this method will Returns the input stream that is attached with this particular socket next output stream and this method will return the output stream that is attached with the soof and next you have public synchronized vo close if you use this particular method it will just close the socket that is a connection the next one is Socket server this will return the socket and establish a connection between the client and server because as it is a socket of the server end it will obviously have to return the socket and establish the connection because that is what it is doing right when a client send the request and if the server will not accept the request then there won’t be any connection between them so that’s what what a public socket accept does and after that you have one more that is public synchronized void close and this will obviously close a server so these are few methods of socket and socket server classes having understood this now let’s move further and understand the client side programming so what do you think a client side programming and a server side programming as so basically when a client wants to connect to a server or if a client has a request at a server should accept and a server wants to send back the response then what happens you need to First write a client program for which you need to send a request to the server and you have to write a server s side programming so that your client will get the response back from the server by accepting the request so that’s why we need the client and the server s side programming and this is achieved with the help of a socket which will help you to connect between a network and which will act as a Interlink to to help you to connect to a network between the client side and the server side to connect to one another machine we need a socket connection as I have already told you a socket connection means the two machines will have the information about each other’s Network location that is the IP address and the TCP port and the java.net docket class represent a class so in order to open the socket what you need to do you need to create a object of a new socket and pass the IP address and the port number okay so that’s how you need to First establish a socket connection using java. net. socket and provide the IP address and the DCP port number and after that you need to communicate over a socket connection and the streams are used to both input and output the data so that’s what is communication with the client where you use the streams to input and output the data and after that you have to close the connection by using public synchronized void close simple so this is all about the client side programming and now I’ll also tell you what is serers side programming and then we will see a demo how to establish a connection and how the client sends a request and how the server accepts a request and how it responds back everything before that I will explain you what a serers side programming so after you close the connection using the client the socket connection is automatically closed explicitly once the message to server is sent and now the first thing is I have to establish a socket connection and write a server application you need two sockets a server socket will wait for the client request when a client makes a new socket and a plain old circuit is used for communication with the client and after that you have to output and communicate with the client through socket so you will use the get output stream method after finishing everything it is important to close the connection by closing the socket as well as all the input and output streams now let’s take an example and understand it’s working so here I have two programs that is the client and the server so first I’ll tell you about the client I have imported the package that is import java.net dostar because I want a network connection that is the reason I’m importing that and after that I’m importing the java.io dostar because I want the input and output streams that is a reason so first what I will do I’ll initialize a socket I’ll initialize the input stream and the output data stream so for that what I’m doing I’m just writing it as private socket soet is equal to null I’m initializing all the variables to null because I want to use it in the later part and after that I’m creating a Constructor for this class that is client and I’m passing the address and the port number that is the IP address and the port number and now what is the first step I have to establish a connection so in order to establish a connection what I’ll do I’ll give socket I’ll create object of a new socket and pass parameters and the arguments as address and port and as soon as it does that I have to print connected and now for that I need to take the input from the terminal so for that what I’ll do I’ll give new input data stream is equal to system.in and after that out is equal to it has to send the output to the socket so that is the reason I’ll use this so any exception then it has to handle and after that I need a string to read the message from the input tab that is the reason I’m using it in this way and one more important thing over here it keeps on reading unless and until over is displayed on the screen so I can write as many number of lines as possible unless and until this over will be displayed on the console so inside that I’m getting line is equal to input. read line and I’m outputting that again after that I’m sending the request I am establishing the connection I’m communicating with the server finally I have to close the connection so I have to explicitly close all these connection that is the input data stream the socket and the output data stream so these are the variables for that and I’m closing all these connection and finally inside the main method I’m just giving the IP address and the port number so this is all about the client but before the client has to say connected I need a server program which has to wait for the client to get connected right so let’s now see about the server program again the same thing but here instead of the output data stream I will get the server socket because I told this is a server site program and I need a server socket for this that is the reason I’m using this and then I am creating a Constructor of this and passing the variable as in Port so I’ll create an object of a new server socket and pass the variable as port and I’ll say server has started waiting for the client then client get connected when I go back and connect the client by passing the address and the port number server says client accepted so after that what I’m doing I’m writing the string unless and until until the over will be displayed so this line will take the input from the get input stream okay and it will read the string line from that and after that I have given input because I need to do input. read line and out. WR UDF so here what I’ll do inside the triy block it will say read the input that is given by the client and then display that after that again I have to close the connection and create an object of a new server with the port number here I’m passing everything here just I’ll give the port number because client is connected to a port and server is connected with both IP address and the port number so that’s how I’m writing this first I’ll execute this server and then it will wait for a client and then I will execute the client and it will say connected and then I’ll start writing anything then server will say client accepted and it displays the request that client has sent and it will rever back the response to the client I hope you understood this now let me first run server so you can do the same execution by opening your command prompt or your terminal but as I’m using eclipse and I have a inbuilt option to execute it like this so I’m doing it on my Eclipse itself so it is telling server started and waiting for the client now what I’ll do I’ll execute the client so run as Java application so it’s telling client accepted so there are basically two consoles it’s telling connected and it is asking me to type whatever I want I’ll go on typing even though I wrote so much it still did not display overr right so it has the ability to enter as many number of lines and you can send as many number of requests that you want to the server right so now when I check the server it’s telling client accepted it’s displaying what all I wrote It’s displaying like hi everyone I will welcome to tra YouTube channel I hope you’re understanding the socket programming in Java and so many things right so yes you just saw first the server started it’s telling waiting for a client and then when I connected with the client it told connected and it just took the string arguments from whatever I was passing from here right and then the server told client accept it and it displayed whatever request I’m just going on typing over there again if I write something else also it will again display in the server screen I’ll just say enter and you can just see that on the server screen right so that’s how it basically works and it will get connected to your client it will send back the response and it will display all the client request and respond back so that’s all about the client and server side programming using sockets in Java so this is how you can actually establish a connection using the socket because it’s Interlink between the two programs that is running on the same network right who is a Java developer Java developer is a computer software developer a programmer who integrates Java programming language into the development of mobile and Computer Applications web development and other sorts of requirements needed at both client and server end now with this let us move on to our next topic which is how to become a Java developer to become a Java developer there are basically three stages in the first stage you have to earn a college degree the first stage requires you to complete a qualification in computer science with a focus on software development this could be any degree related to computer science and information technology or you can either choose Bachelor of engineering or Bachelor of technology in the field of computer science or information technology in the next stage you need to get certified with Java the second stage requires you to complete an online or an offline course specializing in core and advanced Java trying out online examples is a good way to develop your specialization in Java and in the third stage you need to get experienced the third stage needs you to look for a placement or an internship program while studying this will provide you with relevant experience as well as guidance and mentorship from experienced software developers and finally you become an experienced and professional Java developer with this let let us move on to our next topic which deals with the Java developer skills overview when you Google for the skills required to become a Java developer you have a lot of skills showing up now we shall categorize the skills required for Java developers into two variants firstly the Java developer skills Technical and later the soft skills of a Java developer firstly the technical skills the technical skills required for a Java developer are he must be well known with core Java the core Java includes objectoriented programming techniques as well as the design patterns in Java abstract classes in Java interface and serialization followed by that we have om omm stands for object relational mapping so in this skill you need to be well vered with hybernate Java persistence API Eclipse link open jpa and after that you have the Java build tools Java build tools are the most crucial skills required by every Java developer some of the most important build tools are maven and gradal followed by that we have web Technologies a Java developer should be well ver with CSS HTML jQuery and JavaScript followed by that he or she must be exposed to Java web Frameworks like MVC play Java server faces and next to the Java web Frameworks we have application containers the most trending application containers are jbos JY web spere and web logic and next we have Java testing tools test NG junit and selenium are the most trending top testing tools for Java and later we have big data the Java developers must be well vered with dbms Hadoop SQL and jdbc next we have Java e components the most important Java e components are servlets Java beans Java server pages and finally the code Version Control which is GitHub don’t worry we shall discuss each and every one of them in detail for a better understanding in our next topic which happens to be the skills required for for a Java developer firstly we shall discuss core Java the most basic and essential skills required to become a Java developer is the co Java Co Java skills are the building blocks of every successful Java developer The crucial Co Java skills that most recruiters are looking for are firstly the objectoriented programming skills object-oriented programming approach introduces class and object style of coding that deals with the concept of inheritance encapsulation abstraction polymorphism the main Moto of object-oriented programming is to bind the data with the functions together and improve the code stability and safety followed by that we have design patterns design patterns in Java are the standard Solutions followed to solve routine problems faced in software designing life cycle every design pattern acts as a blueprint that allows customization to solve any given design problem in any code during the development of software modules followed by Design patterns we have abstract classes in Java abstract classes in java help the developers to achieve abstraction abstraction in object-oriented programming reduces code complexity by showing the relevant details of the objects used in the program and enhances code readability followed by abstract classes in Java we have inheritance Java inheritance is an advanced level of achieving abstraction in Java programming language the interface in Java enables a better and efficient way of achieving abstraction followed by this we have serialization serialization in Java is a concept that deals with a conversion of objects into a bite stream and transport the Java objects from one Java virtual machine to another and recreate them to the original form serialization enables developers to meet various factors in real-time programming with this let us move on to the next stage where we shall discuss about the Java build tools depending on the task assigned developers need to use a specific tool for every particular task for a Java developer understanding the terminology of build Tools in Java comes handy as Java build tools are the programs that automate the creation of executable applications build tools that one must know are Maven and Gradle followed by this we shall discuss the web Technologies web Technologies provide a simplified and faster way to create Dynamic web content having a better grip on web technology is important as it helps to understand the technique of establishing communication between multiple computer systems via markup languages it gives a way to interact with host for the information like websites and it involves HTML jQuery and CSS at the most followed by this we shall discuss Java web Frameworks to build a mobile application with best features and limited code Java web Frameworks are the best-in-class solutions they support the development of web applications including web services web resources and web abis in a standard way to build and deploy web applications on worldwide web some of the best-in-class Frameworks are spring MVC and play followed by this we shall discuss Java application containers Java application container is an application program in which the software building blocks known as the compartment can be run application containers are designed to provide multiple functionalities such as to provide user authentication logging or establishing a connection with the database topnotch application containers are JBoss and jti with this let us move on to the next stage where we shall discuss the Java testing tools Java Tes with an interface where you don’t have to actually launch your application onto a web browser this support multiple test activities from planning requirements creating a buildt test execution defect logging and test analysis some of the major testing tools that a Java developer should be well versed with are selenium and test NG with this let us move on to our next topic which is Big Data big data has become a trillion dollar industry and hand handling such a huge amount of data has become close to Impossible by traditional data processing software learning Big Data technologies will help the developers to analyze and extract information from complex data sets some of the powerful Big Data Technologies are Hadoop and Spark now with this let us move on to the next topic for our discussion which is Java e components Java e components or Enterprise components the Java Enterprise components enable developers to write server site applications one of the most most common challenges that an Enterprise developer faces is to how to handle requests coming from web-based clients to ease this challenge Java Enterprise provides the Java serlet and Java server pages apis that enable methods for logging activities using cookies on a uses browser and simplify these kind of issues and provide the best solution some major Java Enterprise components are cets Java beans and Java server pages with this let us move on to the next topic which is code Version Control control code Version Control allows developers to keep track of their work and helps easily to explore any changes they made in their data coding scripts comments Etc that is much smoother and easier to implement with this let us discuss the second stage of java developer skills which are the soft skills soft skills are people skills communication skills character and social intelligence among others soft skills enable people to navigate their environment work well with others perform well and achieve their goals now the first soft skill that we will discuss today is the communication skill good communication skills include active listening where you have to pay close attention to the person who is speaking to you and try adapting your communication style to your audience and convey your message in a better manner followed by this the next one is open to learning knowledge is never ending successful Java developers should always be open to learning new upgrades in the technology and keep themselves updated the next one is teamwork teamwork is a collaborative effort of a team to achieve common goal and to complete a task in the most efficient and effective way the next one is work ethics work ethic is a belief that hard work and dedication have a moral benefit and an inherit ability virtue or value to strengthen character and individual abilities the next one is interpersonal skills interpersonal skills are the qualities and behavior of a person used to interact with others properly the better the interpersonal skills the better the growth in man’s career now with this let us discuss the last part of this session which is salary details of a Java developer Java programming language is one of the most demanded ones in the it Industries let us now discuss the salaries offered to Java developers as you can see for a beginner the salary package lies between 3 to 8 lakhs per anom and similarly for an experienced candidate 8 to 13 lakhs perom and the maximum salary offered to a Java developer who is experienced for more than 8 years approximately 28 lakhs perom or greater than that as well the projects that I will be discussing today are ATM machine which is based on core Java Concepts text editor that is based on swings and applets and the signup form which will make use of servlets and JSP Concepts without wasting any further time let’s get started First ATM machine all of you know what is ATM and how it works but implementing the same using cach our Concepts is something interesting so let’s see how to do that here we will be developing the code for user account details bank account and then further we will perform few operations like deposit the amount withdraw check the balance Etc basically it’s based on code Java Concepts and it’s developed using Eclipse so all that you require is your Java install that is your jdk and JRE and you just need to have Eclipse installed on your machine let me tell you one important thing that is I have set the Privileges for account number and password that is you can see on the screen that I have given two account numbers and passwords while running the code if you give the account number and password as mentioned on the screen only then you can do the further operations else it will just display a message saying wrong account number or pin now let let’s see how it works and how the code is being developed so basically when you run the code this is how your console appears and it asks me to enter the account number that will be the one which I mentioned on the screen and the password is going to be this thing enter so when you enter this it will ask you to select the account you want to access that is whether your checking account your saving account Etc so first I will say type f that is checking account so in the checking account the operations that you can do is you can whff the balance you can withdraw funds you can deposit the funds so let’s see how much balance is there in my account it’s zero is it now I want to deposit the funds in the checking account so how will I do that again checking account option three which is deposit funds so it’s asking me amount you want to deposit from checking account so I say I want to deposit some 10,000 enter so you can see here the new checking account balance is 10,000 correct again I will choose checking account and now I want to withdraw the amount from the checking account so I’ll say option two and I’ll say I want some 5,000 from that account so you can see here I withdrawn 5,000 and the new checking account balance will be 5,000 so this implies I withdrawn 5,000 from the account again I will choose option one I will view the balance so it says 5,000 is remaining correct now we’ll go to savings account and again view the balance and do the same operations balance is zero again same thing I want to withdraw some funds like say 4,000 it says 4,000 and now I want to withdraw some funds say some 5,000 let’s see what it says when the amount that we want to withdraw is more than the amount which is in the bank account it says balance cannot be negative correct so when you try to access more amount than which is there in the bank account like the savings account it says balance cannot be negative similarly you cannot just enter the negative values into this now say I want to just withdraw some 2,000 and it says the new saving account balance will be 2,000 so basically this is how you can do operations like same way you do it on your ATM machines and now I want to just say I want to exit so suppose if I enter invalid account number say I’ll enter something like this and I’ll give the pin something it says wrong customer number or pin number that is I have set the Privileges only for those two account credentials so that I can access using only those two numbers you can even include more account numbers and pin I’ll show you how to do that so basically this is all about the working of ATM machine let’s take a look at the code now so here in edura one package I have created three Java files one is for account one is ATM and one is option menu so now let’s see why we have three files for ATM and how it works first we’ll see option menu so you can see here option menu extend account class that is this Java class I’m just reading a money format that will be in decimal and then I’m creating a hashmap of integer and then I’m creating a variable of type hash map and giving a reference for that so basically the first thing that I want to do is I want want to validate the login information that is customer number and pin number as I have already told you that I have set the Privileges for two numbers that is these two account numbers and pins if you want more you can just copy this data. put paste it and you can change like whatever you want you can add more number of privileges like this so the first step when you execute it will just say welcome to ATM project and enter your customer number correct so first what you will do you will enter either of these customer numbers and then it will ask for pin number I’m reading the customer number and pin number using menu input. next int so in case if you enter some characters it has to catch an exception correct so what it will do it will say inv valid characters and it will ask you to enter only numbers because I have set the Privileges only for numbers so in this for Loop I’m trying to check whether the entered customer number and pin number is correct or not if it is not correct it will print saying wrong customer number or pin number that you already saw in the output so after that if the entered credentials are correct then this method will execute that is the get account type in this get account type I have mentioned these choices that is select the account you want to access checking account saving account Choice Etc and then using the switch case I’m using get checking get saving methods if you press exit it will say thank you for using the ATM bu and and the default choice if you press the choice that is not correct it will say invalid choice so here is the get checking and I have defined the method over here that is get checking so what all we had in the get checking method view balance withdraw funds deposit funds exit correct now I’m reading everything that is account balance using money format withdraw input deposit input and everything will be in get account type method correct and obviously case 4 will be your exit so it will just print saying thank you for using the ATM by now similarly for savings account that is view balance withdraw funds deposit funds Etc again we are using switch case to print the saving account balance saving withdraw input saving deposit input everything so this is all about your option menu class now we will see what is there in account class so here it is just require to set the customer number and set the pin number that is first to set the customer number and return customer number and then you set the pin number and return the PIN number so basically account class is to retrieve all the details that is being set in option menu that is your checking balance saving balance your withdraw input deposit funds everything that is being mentioned in option menu class so you can see here that I have checking balance method and I’m returning the same and again I have saving balance so here you can calculate the checking account withdrawal that is checking balance equals to checking balance minus account and then it will return the same that is we saw that when we want to withdraw 2,000 from 4,000 it will say the new saving balance will be 2,000 similarly the same thing will be here that is saving balance minus amount like that so similarly you can calculate the account deposit the saving account deposit that is the balance plus the amount that you enter suppose say the saving balance is zero and if you enter 2,000 so it will add and it will return the balance similarly the same thing with this as well that is account withdraw input and customer saving account withdraw input as well and then it will just check the deposit amount same thing for savings as well and I have initialized all these variables as private because I want them to be local to the class so this is how it works and now coming to the last file that is atm. Java so here this class extends the option menu and option menu extends account so what ATM should do it should obviously extend option menu and I’m just creating an object of option menu and just trying to get the login from option menu so when I execute again it works same it will last same thing again if I enter something it says wrong customer Pinner account number as I have told if I enter something else apart from the numerics it says invalid characters only numbers that is the exception that I have thrown so this is all about the ATM project I hope you understood it now let’s move further and see the text editor so here I’m making use of java swings that is I will be using J text area menu bar J menu bar Etc and all the menu items will have action listener to detect any action and as I have told it makes use of java applet and swing Concepts and similarly all that you just need is a Java installed and the eclipse ID we can say like we can perform all the operations like we perform in your wordpad or your Notepad and it’s very efficient so now let’s see how it works so first let’s execute and see how it works and then I will explain you the code so you got a text editor here you can see you have file and it consist of all the operations like new open save exit and you have edit option like cut copy and paste you have font like bold plain italic and you have size like up to 28 and you have font type that is Roman helvetica IAL corer Century Etc and also you can set the background and foreground color and you can undo and redo the operations now let’s see each of them how it works so I want to say like welcome to edure YouTube channel and enjoy learning now suppose I want to change a font of this I’ll capture everything like control a and then I’ll give font like bold so you can see here it became bold say suppose I want it to be italic you can see even that became italic as well I want to increase the font size like 28 it got done I want to change the font type to Century even that happened say I want helvetica I want times Roman even that happened say I want a color to give for that like background color in something like say I want red okay you can see the background color is red so I want to remove the background color now now we change the font color to be say I want to enter some more text like I did as Java projects and I want to undo that I can just press contrl Z and it goes on removing all that thing and control 5 for whatever I have deleted you can see it’s going on also you can create a new file that I just did also if you want to open something that is is already present you can open from the libraries also if I want to save this I can save it like however I want in the downloads or in desktop whever you want you can save the file by giving save it it will be saved I can enter something and again if I want to save it it will be saved like applet one similarly you can even check that whether it is being saved or not so in desktop you can see it is being saved as applet and applet one so basically this is all about your text editor as how it works like you can perform all the operations like edit font file color undo redo everything Etc sounds interesting let’s see what is the code behind this now so here you can see a undoable text area that extends the text area and Implement State editable why Implement state editable is because we have defined the editable class in which we can edit all the options like I showed you on the text editor so you can see here I have given the key State as undoable text area key and then I have declared the current edit and undo manager and the first undoable text area I have defined it in various ways the first thing it is zero parameterized and then in the next I have defined some parameters like string and I have given it in super as well and then I have used rows and columns and then I have used string along with rows and columns and then string rows columns and scroll bars as well so in various ways I have declared this undoable text area Constructor so if in case something happens you are stuck and you cannot undo some operations that you have perform then automatically the exception will be catched here and next same for redo as well if you cannot perform any redo operations then it again catches an exception you might be wondering why it need to catch an exception correct it’s just because to maintain the normal flow of execution so that your further operations will not be interrupted and next to store the state and restore the state that I have given I’m using the key State as a variable and for the snapshot method I’m giving like condition if the text is being changed then use undo and add edit to edit the current state and now I’m declaring the init undoable and giving some new reference variables for undo manager and current edit and I’m adding the key listener for new adapter and then I’m using the focus listener to focus the event that was being captured and similarly for text listener as well if the value is changed it has to say that the text is being changed and now you can see there’s a text editor class which undoable text area has extended in this I’m declaring few parameters like frame file dialog font Etc and now here comes the text editor parameters like say corer at style it’s size everything and here I have used the option menu bar that is times Roman helvetica aial Century Etc and I have given one one parameters for each of them and then using the add method I’m adding all of them like appending it to the menu and we have to use the action listener to detect any action that is being performed so similarly for bold plain italic menu that all we saw and again using action listener we are calling that event and then I’m declaring all the sizes over here that you saw L till 28 and appending it using the ad method and to detect them I’m using add action listener even for new open save exit again using action listener for all them and for cut copy paste edit options as well for background and foreground color and for color for undo Redo for everything it all Remains the Same and then I’m implementing the action listener class to invoke that method correct like for undo and redo and similarly for file name Untitled for select menu everything so whatever the options we have seen here they all have to implement the action listener class so you can see I’m detecting save else what happens the action will not be detected you can see again for exit as well for read file everything and even cut copy paste for everything we have to use action listener classes even the size class implements action listener like for size 10 size 12 for 14 16 18 similarly till 28 that’s all and then I’m extending an applet class and implementing the action listener even for this in bold I want corer font everything and even in plane I want the same thing so as you can see if I type anything here so it is by default corer correct so that’s what is here that is in get action command in bold plane and italic whatever it is the first thing is corer correct and then comes the type that is times Roman helvetica corer aial Black Century everything and then again I’m am choosing the color that is the background or foreground colors and then I’m implementing the main method and setting the size of the text editor to 800 and 600 so that’s how it works like you can perform any operations on text editor and do rest everything now let’s move further and see the sign of form which is developed us using surf GSP and web development tools like HTML CSS Etc so I’m just going to create a basic sign of form and all that you need is servlets Java server Pages Apache Tomcat server and cascading style sheets and eclipse for Enterprise Edition and it’s very easy to develop as I’ll be showing you how so first let’s run the output and check and then I will explain you the backend code make sure that you run on server my Tomcat server is starting now so this is the sign up form that we have created you can enter the name I’ll give some email like 13 or older is my option so here in the sign up form you can see your name email password your age I’ll give some biography like I am a footballer singer swimmer Etc like just my Hobbies I can give my education everything you can give like it’s not mandatory that it should be the same thing and the job role have so many options in web mobile business and others so I’m very interested in iOS Developer so I’ll give that and interests are in development design business everything and when I sign up it says hi Cris welcome to Eda because the sign up that I have done got saved so where will be the output like or where you can see the details that you have entered in the back end in this console tab you can see the details that you have entered that is your name your email ID your password your over 13 your biography your job role your interest Etc so that’s all was there in the form so you can see here in this form these are the basic information that is being given so now let’s take a look at the backend code in the sign up I have created five files like HTML for creating a form CSS file to add some beauty or makeup to the form that we have created we have seret class files like request post form and person one. Java class and then to retrieve the data from the servlets we have the JSP file and also we have the web.xml file to add the mappings to the serlet and we have JSP file to retrieve the data from the servlets so for this front end that is your sign up

    form we have used HTML you can see here that I have linked with CSS as well that is for giving some makeup to the file that is you can say beauty to the form that you have created I’m giving form action as request post form and Method will be post so here you can see in the label you have created like name email password age your under 13 or above 13 whatever it is and then for your profile or your biography you have created one more label tag and giving the text ID is equal to bio and name is equal to user bio and for the job roles there are so many so you have created a first label for job rooll and the first opt group label will be web in web you have so many options and again you are closing that and opening one more op group label that will be your mobile your business your SEC your other so same things and then for your interest like this one you can can see you have check boxes over here correct for that check boxes we have given the Class Type value everything like for development for design for business everything and last it will be the submit button that will be your sign up so this is how your index.html file will be and now coming to main. CSS file for each and everything I have given the paddings like the font family the color the maximum WID the margin background everything even for the first thing that will be the H1 I have aligned it to Center you can see here I have aligned it to Center and that is an H1 even for that I have given the specified color and the F family as well and coming down for form as well and for all these things I’m using a select type for background border font size height margin outl padding B color everything even for the Box Shadow as well you can see there’s a shadow for this box it’s not just a simple one and even for the check boxes for buttons that is present as well for field set for legend TXS for label for number TXS everything so this will be your main. CSS now let’s take a look at the serlet classes so first what I’m doing here is I’m extending a HTTP serlet class correct and I’m using a do Post method and declaring parameters for request and response and I’m requesting all the parameters that is your username email password age biography job and for your user interest again the same thing and then I’m creating a object of person and trying to retrieve all the details and then I’m trying to request and set the attribute of my person and then using the dispatcher I’m trying to request the get request dispatcher from JSP and obviously in the dispatcher forward you will be having a response and request both of them so as I have created the object let’s see what was there in the person in this it’s nothing but it will retrieve all the details that is I have made everything as private because I want it to be a private to the person class and this class is called as a being class I’ll tell you why I have made a Constructor of person one and then using the parameterized Constructor I’m trying to reference and retrieve all the details you can see like for get name method I’m returning the name and set name I’m retrieving again for get email and set email methods for get password for get dat set AG for Bio job roles interest everything this is nothing but uh defining a Constructor and retrieving the details and request post form is essential to send a request and response and then person one. Java is essential to retrieve the details and then as I have told it is requesting a dispatcher to login one so in the login 1. JSP class I have imported the bean class that is edura person one why because edura is my package and person one is my class and then I’m giving the scope as request and similarly as I have imported the bean class and similarly to redirect it to the next page I have used a small icon and you can say that it says welcome to urea so this is all about the GSP and then now coming to the last file that is web.xml you have to add mappings to the file you can give serlet name like anything but make sure you give the serlet class name very properly that is it should be like edura that is your package and your request post form which is your class name this should be very correct and then for URL pattern that is request post from URL this should be same as the one which you give in form action tab that is your request post form URL or else it will result in an error so this is all about the sign up F I hope you understood how it works so let’s just start with something known as the basic questions which are asked to the candidates in the entry process so the very first is the difference between three terms that’s like jdk JRE and jvm so as a Java developer you must have a fair knowledge what exactly are these three terms right so guys let us understand now so Java virtual machine is basically an abstract machine it’s a software which is already provided by the platform so it provides the runtime environment whenever you are writing the Java source code it gets compiled into the bite code so jvm is that there which is going to execute your bite code now other than that you got JRE which is a runtime environment now it’s basically where JV is going to use the JRE so JRE set of libraries required by the jvm in order to execute your code jdk is one development kit required by the Java developer so it consists of various apis for you through which you can write your code so just for an instance right so jdk will contain your various packages coming into uh for example java. iio java.util all these API is the set of Library the information that comes from your Java development kit other than that it has various tools also available for example Java and Java C right so Java C is a tool which is used for compilation whereas Java is for the execution of your bite code so these tools are also part of jdk so let’s switch on to the next question what is synchronization this is one of the hot favorite questions whenever you talk of threads so we do have this term known as synchronization so synchronization is something where multiple threads if they going to access the same object right so there is only one resource which multiple threads they want to access you need to come up with something known as synchronization so guys what is synchronizations here we’ll take up one example let’s say we want to book a movie ticket and it’s going to be one seat which has to be booked by multiple people right so let’s say one corner seat in the last row has to be booked by multiple people so multiple people over the web will treat them as multiple threads they are looking onto a single object on a single resource that’s like the last row last seat right so whosoever will first of all try to block that seat will be given an excess and no other person can thereafter block that seat so synchronization is very much line to this concept when you got multiple threads which are going to work over a single object you need to use a keyword known as synchronized with the help of which you will be able to acquire the locks on your objects so synchronization is a process which going to keep all the concurrent threads in the execution to be in a synchronous mode one after the other the lock Acquisitions going to come up into action synchronization will avoid the memory consistency errors due to any inconsistent view of the shared memory let us understand the difference between processes and threads now so synchronized word came up and we started off something known as thread so let us understand what exactly is process and what is a thread now so guys in order to come up with process and thread let’s just have one visual coming in let us say you got this layer as an operating system right so we’ll say that this is one operating system it can be any OS now right it can be Windows Mac Linux any of the flavor so when you’re going to run the program what OS will do OS will create a process right so this process will be created by the operating system whenever you are supposed to execute your application so other than that what it does is it will all also give you some you know storage space where Ram will come into action right so further the ram can have various data structures coming into it for example you can have the data being stored in the form of stack or the Heap right so that’s like the storage concept let us say I got one program which I need to execute so whenever I’m going to execute my program so the running instance of your program over the operating system is nothing but it’s a problem process within the process we have one guy known as main thread so main thread is what this is the guy which will execute some instructions or you can say some sequential tasks which you mention in your main method right so whatever the statements you are writing in your main method they are nothing they are these tasks which are executed one after the other via main thread within the process so guys a process is running instance of your program over the operating system and a thread like a main thread is running within the process so we must understand threads are available within the processes and processes are running on top of operating system there might be a scenario that a particular task like I’m just going to highight this task over here so this might take a longer time right so let us say this particular task will take a minute or so because it’s going to fetch the data from the server now so in this context right so these three tasks they will be blocked right so when I’m talking about these three tasks the yellow States so they going to get blocked now so what we can do we can bring down one more thread and we’ll say that this is one user thread or child threat so we got various other terminologies for this guy right and will offload the heavier task over here of the main thread to the user threa in this way what we are able to do is we are able to Outsource the heavier task to some other trade and we will be coming up with a concept of concurrent programming that is what we want to achieve now right so let us understand what has happened over here guys so let’s see this part now so process and thread they are two different concepts but we must understand now any executing instance of a program on top of your operating system is nothing but it’s a process so on operating system we got multi-processing whereas thread is something which is a subset of the process now so within a process you got multiple threads so multiple processes on OS level multiple threading on process level when you talk about the communication so one process when it wants to interact with the other process we call the processes as interprocess communication where threads within a process they can directly interact with each other when we talk of the control statement so processes can only EX exercise control over the child processes whereas threads can exercise considerable control over the threads of the same process any change in the parent process will not affect the child process whereas if there is something where the main thread is getting affected the other user threads or the child threads they might get affected if you talk of the memory structure so processes they run in the separate memory space each and every process will have its own memory space space whereas threads share the same memory space which is being allocated to the process by the operating system so guys once again process is controlled by OS whereas threads are controlled by the programmer in a program right or you can say by the process itself if you talk of the dependency level processes are independent where threads they’re going to get dependent because if you can notice this image right over here so we created this user thread and there is a dependency that this task has been outsourced to this user thread by the main thread so dependency of the main threat over the user threat relies and vice versa right so it can be based on our Logics even so let us move to the next concept now so the next fundamental is what exactly is a repper class now this is also one of you know the hot favorite question which is always asked what is repper class what is meant by the term boxing or the auto boxing so let us see this part guys so repper classes they are the way where we can convert the Java Primitives into the reference types that’s like the objects so each and every primitive type has a repper class defined and it’s in by default they are available in java. L package you need not to import any package so for Boolean we got Boolean right for car we got corrector and for INT we got integer for double so on and so forth guys so repper classes are the means by which you are able to convert your Primitives into the references so let us understand what exactly is this conversion of Primitives into references let us say I got this i as 10 so we call this guy as a single value container because I can contain only one value it’s not an object now it’s a primitive type you want convert this primitive guy into an object you say integer I sign new of integer and you mention this guy I as a part of Constructor so what we have done we have boxed so it’s like constructing the object now constructing the object so that’s what we have done guys over here right so this particular statement over here is known as boxing on the other way around when you say in J assign IR do int value so this concept is known as unboxing right extracting the value from object so you must understand this concept over here boxing versus unboxing which is achieved with the help of rapper classes other than that you can also say integer SCF assign I so this is known as autoboxing right so this instruction is translated internally by the compiler as in integer KF assign new of integer where we got this guy known as I right so this is auto transation you need not to write the whole statement so you just mention the partial statement or the implicit statement we call call it as autoboxing and if you write down let’s say l assign K right so this is known as Auto unboxing so important question again what is boxing versus unboxing and thereafter what is auto boxing versus Auto unboxing so the major reason why we do boxing or unboxing is to convert Primitives into the objects now if you’re talking about objectoriented programming structure right so we are always thinking of objects now so if you are able to convert Primitives into the objects it means that you are strictly and truly following object oriented programming structure so we got one way of doing it and now you can also pass by reference The Primitives so that’s one more uh way of coming up like why you do a boxing part so we can pass the variables by references now right because here you are getting the references to the objects all right so let’s come back now to the next question number five that’s like what are the keyword final finally and finalize and what is exactly the purpose of it so there are now the three different keywords right so even the final term is something which means the last thing but they have a different meanings in different context now now final is a kind of a constant so if you mark a variable as final you can’t change its value so here if you can see a very small snippet right so a is 1,000 and you try to change its value the compiler will not allow you to do so so final is a keyword where you are able to mark your variables as constant you can also Mark your classes to be final in that context you cannot inherit from that class you can also Mark the methods to be final in this regard your methods they will not be overridden so final can be used with three of the concepts final as a variable final as a class and final as a method finally is the other context where we use it with the exception handling techniques now so what exactly is the place where we will be coming up with finally so many of times when you are writing the exception uh hand code with a try and a catch Clause there might be a scenario whenever the exception will come the statements mentioned within the tri block they will not be executed below the statement where the exception will come right so let’s say there are 10 instructions in the try and on the fifth instruction in the try you get an exception so below 6 7 8 9 and 10 instructions they will not be executed you need finally block to mention such instruction which must be executed either exception comes or it does not so finally is that piece of block which will be executed whether the exception is coming or it is not coming for example if you want to close the database connections now exception is there or not but we would like to close the connection at any cost right so you mention all such statements in your finally block so the next part is finalize so guys finalize is a kind of a Destructor so whenever the object will be removed from the memory by the garbage collector so this finalized method will be executed so you can explicitly try out this example where you can say system.gc right and you need to just mention your references to be pointing to the null and thereafter your finalize will be executed so we can just have a small example on this right so what I can do I can just demonstrate this so let’s say we got this class known as finalize demo right and I’m just going to make this method finalize over here so I’ll say ciso object finalized right so here we go now what I’ll do I’ll create the object of finalized demo so I’ll say finalize demo FD reference assign new of finalize demo so here we created an and thereafter I’ll say the ra to go as null it means it is not pointing to anything now so later I’ll say system. GC so GC is called to the garbage collector and it’s going to destroy the objects which are no longer required so now you can see over here so let me run it as a Java application it says object finalized why is it coming because it’s acting as a Destructor call so anything which you want to do before your object is removed from the memory right you can write it into the finalized Method All right so let’s go to the next concept it’s a difference between string buffer and string Builder right so before we jump onto the string buffer and string Builder we must understand they are both similar Concepts when they are compared with the string class so string is is immutable version whereas string buffer and string Builder they are the mutable versions right so let me Define this part what is meant by mutability versus immutability consider you got this string sdr1 which is a new string having hello and you got this one more guy known as string Builder you say Builder sign new string Builder and you mention hello similarly you say string buffer let’s take this guy as buffer you say buffer assign new of string buffer again hello so I have taken up these three strings let us say St Str Builder and buffer and now what is next I will be appending the data in these strings now so you say St str. concat so in string you need to say the function function as concat so I’ll say hi and similarly let us concat it into the buffer and the Builder so I’ll say Builder do append so in the Builder the function which we use is append and same goes with the buffer so buffer and Builder they are quite similar now later when I’ll do a ciso on St Str and similarly the Builder and the buffer part let us say Builder and then finally the buffer so the buffer here we run the code you see one major difference over here is that you get hello so nothing happens when you concat the data into the SDR right but in case of Builder and buffer the data is concatenated this is why because strings are immutable whereas buffers and Builders they are mutable this is also one of the very B beautiful query first of all so guys buffer and Builder they are the mutable sequences whereas string it is immutable you cannot change it even though you are going to perform an operation so what is happening over here whenever you concatenate the data you will be getting a new string right so you need to store the data into a new string so concatenation results the data coming out as a new string the old string string is not changed so significant difference between buffer and Builder now we’ll be discussing string buffer operations are thread safe so it means that by default when you are going to use buffer it will be synchronized you need not to worry on top of multiple or concurrent threading now whereas Builder it’s not thread safe string buffer is to be used when multiple threads they are going to work on the same string whereas Builder is supposed to be in a single threaded environment other than that Builder is quite better as compared to buffer over the performance part because there is no synchronization overhead so you can just try to write out this nippet over here which will be able to help you to come up with the timing difference and you will be able to see that Builder will outshine the buffer in terms of performance all right so let us understand the difference between Heap and and stack so stack and Heap they are two memories which are very much famous when you talk about the Java runtime right so Java runtime the ram we got these two major data structure stack and he so first of all let us see how this stack and Heap uh will come into action right so let us take one example guys so I’ll just remove this Snippets all right so let us understand the stack and Heap into action when we are talking about about the ram allocation so this Ram will be divided into two parts the first part is the stack and the other part will be the Heap so this circle is representing the Heap memory so both of them are a part of ram now right so both the data structure they are part of ram we got stack and Heap so let us understand this point of stack and Heap how the data will be stored and where it will be stored I’ll just take up one basic example from the eclipse now whenever you will be creating an object so for example finalize demo I said FD ref assign new finalize demo this is known as object construction statement right so this is object construction statement so here what we are doing we are trying to create an object with the help of new operator and to be very much uh more precise I’ll take one class like point right and we’ll Define some X and Y as attributes within this class point right and I can have one function like show point which can just display me the data of point so I can say point is plus the value of x let us say a hyphen plus y right so that’s like a point now so how I’ll be creating the object of point now same way you created the finalized demo you say point P1 reference assign new of point so how this statement will be represented in the memory so guys for this statement to be represented in the memmory what will be going into the stack what will be going into the Heap let’s have a close look onto that part now P1 ref over here will reside into the stack like this right so this guy is P1 ref it will be created on some memory location so let us assume it’s 4,01 it will be a hexadecimal address but I’m just assuming it to be one address over here as 4,01 so this memory location this container P1 ref is created out over here what is next the next part is an object which will be constructed in the Heap which will have two compartments the first compartment is your attributes that’s like X and Y and the second compartment goes like the methods show point so this is like the attributes and the second compartment is for your methods now what is happening over here so this object has no name now right so this is having no name on the Heap you don’t have any name for this guy but you can definitely have an address over the M let’s say this is 601 so this object is constructed at 601 so P1 one ref will have the address of your object and this guy P1 ref is doing nothing but it is pointing to this guy 60001 so we say T1 ref as a reference variable because it is referring to the object available at 60001 it’s not a kind of a pointer now right in order to prove this uh prove this concept we can just say C out of P1 ref when I will do a sis out of P1 riff let’s say P1 riff is plus the value of P1 riff now I’m going to run the code you will understand that P1 riff over here is showing me some address of an object of type point right since we got in the package cod. Eda so it says that code. edao at memory look so this is the memory location 7852 e922 which P1 ref is holding for your object so now once we are good to go what is available in stack what is available in Heap for this instruction right so guys anything for which you use the new operator that will be constructed in the Heap so Heap is that memory location where we use use the new operator and new operator is used to create the things dynamically when I use the word dynamically it means at run time it means anything which is constructed dynamically at run time in your program it will reside in the heat right so that’s what the new operator will help us to do now let us have some more differences between stack and heat so guys stack memory is used only by one thread of execution whereas Heap memory is used by the all parts of the application if you talk on the excess level objects stored in the Heap are globally accessible whereas stack memory cannot be accessed by the other threads memory is like Leo whereas the Heap memory is basically the tree structure now right so the memory management is based on the generation Associated to each object so here what we try to store the data is in the form of ke value pair so each and object will have a hash code now so that acts as a key for your object then you talk of the lifetime so we got existence until the end of execution of the thread whereas Heap memory will live from the start till the end of the application execution and your garbage collector can anytime free this m space if you talk of the Usage Now when an object is created over the stack so now please guys in case of java every object is dynamically constructed so there is nothing like objects being constructed in the stack right so we are just giving a general overview of Heap and stack over here not typically inline to your Java these are the general differences now in Java your each and every object will be constructed dynamically in the Heap space not in the stack space so in the stack space you will be getting the references to the objects now right so guys whenever an object is created it is always stored in the Heap space and stack memory only contains the local reference variables which are pointing to the objects in the heat so let us move towards the next basic question that’s like the difference between the array list and a vector so these are the two data structures these are the two collection apis in the Java do util package so both of them are quite same the very major difference is that array list is not synchronized whereas Vector is synchronized so guys when your data structure is not synchronized so the performance will be a bit more better as compared to the other one so that’s why your RLS will behave faster in terms of performance as compared to your vector if an element is inserted into the RLS it increases the size by 50% whereas Vector they don’t do it so but they just double the the size of its array right and rs does not define the increment size whereas in the vectors you can Define how much size the vector should increment when it has reached its limit other than that rlist can only use iterator for traversing whereas in Vector right except hash table Vector is the only other class which can use enumerations as well so you can enumerate over the vector with the help of enumerations we can also you know iterate within the vector so that’s that’s like some of the differences which are highlighted over here similarly the other collections uh hashmap and hash table so this is again a relevant question which is asked in the interviews many of times as a basic question so hashmap and hash table again the first difference is the synchronization right and your hash tables they are synchronized whereas hash Maps they are not synchronized hashmap allows a null key and a null value whereas hash table will throw a null pointer exception if you’ll try to add a null key or a null value other than that since a hashmap is not synchronized it’s going to behave faster as compared to the table and we can make the hashmap as synchronized by calling the collection synchronized map API so it’s not like that we cannot Mark the hash M synchronize we can come up with our own Logics as well right so guys hashmap Traverse can be done through iterator whereas table can be enumerations as well as iterator hashmap will inherit abstract map class whereas hash table is totally working on the dictionary class now the next concept is the difference between equals and equal to operator now this is one you know as a new Java developer many times we’ll do this thing we’ll do this big mistake by comparing the strings with the equal to operator so unlike your other languages here you need to use the equals method to compare the strings now right so guys let us understand this concept first of all I’ll just try to demonstrate it by writing one example so let us say you got one string s Str one as new string so just recently we had this discussion that this is an object construction instruction so s Str one is a reference variable whereas the string object will be constructed in the Heap so St str1 is having one address right so let me write down this thing Str str1 is a reference variable it is having the address of an object similarly I’ll create one more Str str2 right so Str str2 is also a reference variable it is having the address of some other object the difference over here which we need to understand is that there are two different objects which are containing the same value hello right so objects are containing the same value hello and S str1 and S str2 they are the two different references and having the different values so that’s why if you’re going to compare it using equal to operator right so it’s going to say Str str1 let me just finish out the program and in the else part I’ll just say not of equal to right now you will observe that s str1 is not equal to Str str2 the reason is very similar because you are comparing references now so double equal to we are comparing references right so that’s one important difference for us to understand now when you need to compare the objects you don’t use double equal instead you use this method known as dot equals right so you say let us compare the strings using equals method so here I’ll say equals and not equals so we don’t compare strings using the equal to operator that’s very risky it’s it’s one big mistake you compare the strings with the equals method equals meth method is not comparing the references it is comparing the values within the objects right so that’s what we are able to understand comparison of references versus comparison of values so you can take up this program guys over here if you can see right just go ahead and try to run this execution we got one more program over here let us predict the output for this program so in the main method you are calling the method one with the input as 11 so you pass this 11 over here from here you are invoking the method two by passing 11 into 11 and in the method two you are passing method three which is division of 11 and then in the method three you are subtracting 11 and finally in the method four you are adding of 11 so guys finally we’ll be getting the result as 11 itself right because 11 11 into 11 it’s going to be 121 divide by 11 it’s going to be 11us 11 it’s going to be 0 + 11 it’s again going to be 11 so try out this program right and see to it if you can get the same output or not so these are some of the basic questions right highlighted by us which are very important and crucial from the interview perspective next we’ll be jumping on to the objectoriented programming structure now this is one key area which company or an interviewer is looking and uh expecting as a developer you should be quite good on these Concepts because the advanced concepts or any of the application development model it relies heavily on object oriented programming structure so let us see what we can have in the objectoriented programming structure so the very first question as in question number 12 is the difference between abstract classes and interfaces so let us see some of the differences again guys this is quite theoretical information right uh but I would like to take one very small demonstration on abstract class versus interface because it is very important to have a very fair knowledge and this concept is linked with your runtime polymorphism so let us understand this concept so I’m going to create one class shape and we’ll have this uh method known as a draw right so let’s have a draw method and you say see so drawing a shape now this is something which even we teach in the traditional language like cc++ I mean Native languages so this is the same concept carried forward we got a shape class and we got a draw method so what you going to do you’re going to create the object of shape as assign or let us say shape s and then say s sign new of shape and then you can say s do draw so the output is very much simple you get the mathod drawing a shape but I want my shape to be be either as Circle rectangle or polygon right so what I will do I will just come up with a class known as Circle so I’ll say this guy’s Circle shall extend something known as shape right and in a similar way I’ll also try to come up with something known as rectangle and lastly I’ll say a polygon so shape has three children now and each and every child will be redefining the draw method with the respective shapes right so here we are now this is very well understood that whenever you’re going to redefine the method so the concept of overriding will come into action right so this is overriding and now if you say Circle C and C assign new of circle and you say c draw you will always get drawing a circle because overriding has come into action what I want to do I don’t want to do this concept right it’s it’s a basic concept now that whenever you’ll create the object of child always the definition of child will come into action as a concept of overriding what I want to do I want to say reference variable belongs to the parent and the object belongs to the child now this is known as polymorphic statement so how it is a polymorphic statement now the guy shape s is pointing to Circle so shape is now behaving as Circle let us assume that now this instruction is equated it’s very well equated the equation is true reason is that circle is also a shape circle is a child of shape so technically their type is same now right so I’ll say s do draw so I’m saying s do draw what you’re going to observe is you get drawing a circle now which is very well true because there are two definitions One belongs to the shape other belongs to the circle and when you say s do draw you’ll be getting drawing a circle so you will be giving the preferences to the children over here and the parents definition they again going to get overridden what I’ll do now I’ll say s is now a new of rectangle right you say s do draw and then you say s is new of polygon and then you say s do draw so every time you are able to achieve s do draw drawing the different different shapes right so this is what we call as in runtime polymorphism the same method draw over the same reference variable is we are able to draw different different shapes dynamically at runtime so guys this beautiful concept is known as runtime polymorphism right and what we need to understand from here is that the guy draw in the shape method is useless having this concept over here that s is a new shape so having the object of shape and executing this guy s do draw makes no sense it’s not a pretty good sense right drawing a shape makes no sense shape has to be Circle rectangle or the polygon so what we need to understand from here is we need not to have the object for this class and we need not to have the definition for this guy draw so you can change the architecture of your program and you can bring down one more class shape mark this guy as abstract and mark the method Within your class also as abstract so an abstract class is a class for which you cannot have objects now you can’t create the objects right so here you can see it’s an error other than that the method can also be marked as abstract which means you need not to have definition so guys when a class is marked as abstract you cannot create the objects but runtime environment will create the object right so see so I say shape is constructed very very important difference which I’m going to highlight today so when you say the Constructor over here right so you are able to create the Constructor it means the objects can be constructed for the shape but you cannot create it runtime environment will be able to create that you cannot create if you will try to create create it it’s going to give you an error right so the same problem statement is solved drawing a circle drawing a rectangle drawing a polygon but before the construction of the child object the parent object is constructed by the runtime environment so we say this is object to object inheritance right so what is happening over here inheritance is happening here from object to object so the parent object is constructed by the runtime environment and then the child object is created so abstract class is a class you cannot create the objects but the runtime environment can do so and it can have abstract methods which have no definition so no definition for this guy who has to Define this guy the child has to mandator define the definition for the draw draw so if the child is not defining the draw method it’s going to throw an error so let’s try to see this so you try to comment out this code you can see now it is giving you an error so if there is n number of abstract methods in the abstract class you need to redefine them in the children now we can understand this part that this shape has helped us to improve the concept of runtime polymorphism but this is again which is like a parent is writing some rules which are supposed to be defined by the children now there’s a relationship known as parent and child relationship so abstract class shape is when extended by the circle it’s a parent child relationship for example the parent is giving is just making a rule that breakfast has to take place at 800 a.m. in the morning Now The Childrens they need to Define this Rule and they can come up with their own definitions okay we’ll come up with a breakfast at 8 A.M but I’ll have uh snacks I’ll have bread and butter or anything right so that’s like a parent and a child relationship and from basic shape class we came to an abstract class shape with some you can say there were some problems over here that we didn’t want the object to be coming up and we didn’t want the raw method to take place right so we achieved a better way of coming up with runtime polymorphism through abstract classes now I’ll come up with the best way so I’ll say one interface shape with the method as white draw so guys this is the best way to achieve runtime polymorphism now so what I will do I will not say extend shape but I will say Implement shape now this is the difference which we must observe over here so rather than doing an extension I am doing an implementation and in the interfaces by default the methods are public abstract so it is by default like public abstract wi draw and in the interfaces if you will observe now so this Constructor is significantly an error it means that neither you nor the runtime environment can create the objects of shape right so it’s an error right so for the interfaces neither you can create the objects nor runtime environment can create objects it means this is not object toob inheritance there is nothing like parent and child relationship over here what is the problem over here this is the kind of traffic police or the police department is imposed osing the rules on us to wear a seat belt and now we need to Define that rule right so traffic police is imposing a rule to follow the traffic lights and it is up to us that how we Define that rule so it’s not a parent and a child relationship kind of a concept it’s not parent object and the child object concept right I hope you got the example and you are able to correlate the fundamental between the interfaces versus the abstract classes now since this method is by default public so in order to redefine you need to always have a higher excess or the same excess right so this is how you have achieved one more refined way of runtime polymorphism right guys so what we have understood let’s come back and now summarize the whole Concepts so abstract class and interfaces there are the two different concepts but we can accomplish runtime POC morphism with both of them one being the parent other one being one implementation right so guys an abstract class can provide complete default code or just the details that have to be overridden whereas interface it will not provide any code so it’s just the signature there cannot be any definitions in the interfaces whereas abstract classes they can have the definition so we can only extend a single abstract class whereas a class can Implement several interfaces so this is not multiple inheritance it is multiple implementation Right Moving ahead now abstract classes can have even regular methods whereas interfaces they cannot have now the instance variables in the abstract class they can exist but an interface it cannot have any instance variables its variables are by default static variables right in abstract class is you can use public private protected so on and so forth whereas in interface by default the visibility is public or none right so it is by default public somehow if you mention nothing it is by default public now so here we can have Constructors here you cannot have Constructor it means that you don’t create the objects but runtime environment will create here neither you can create nor the runtime envirment can create abstract classes are faster as compared to the interfaces they slow because it requires extra IND direction to find the corresponding method in the actual class right so that’s what we got some of the theoretical differences after the Practical implementations so polymorphism I hope is now very well clear right so guys polymorphism means more than one forms where a mobile phone can serve the purpose of camera remote MP3 player and alarm clock so on and so forth right so polymorphism can can be characterized into two parts one is at the runtime one is the compile time so guys runtime polymorphism is where you are coming up with this parent and a child relationship again we saw this math example shape Circle polygon and rectangle right so what what happens in the runtime polymorphism you create a parent class and then you create a child class child class must override the method of the parent class and thereafter you do an upcasting by saying the reference variable of the parent can point to the object of child and whenever you will execute the method you’ll be able to see the method execution happens for your child so what is the difference between overloading and overriding so guys overloading is concept linked with a compile time polymorphism whereas overriding is linked with the runtime so in the definition part math method overloading means that the same method name should have different inputs or the different arguments right every time they must be different now so in method overwriting the signature should exactly be same so here return type will not make any difference but here everything should be exactly same if you talk of the behavior method overloading is basically adding or extending the methods Behavior whereas overriding is changing the existing behavior of the mathod now right so here we are trying to add the functionality here we are trying to change the functionality so compile time polymorphism versus runtime polymorphism different signature versus same signature and when you are talking of The Inheritance it may or may not need inheritance in method overloading whereas overriding is only linked with your inheritance so this is one significant difference guys if you can see right so this we have already observed the overriding part but in the overloading part if you can see over here the add method with the two inputs of type integer and two inputs of type double so this is one uh significant example right so guys uh this is like overloading and this is like overriding right so you can just I think this is done wrongly so overriding is this example overloading is this example all right so if there is any method which is private or static so can you just try to overwrite such a method so guys the answer is no you cannot do so so private method will not be inherited so it makes no sense to override it or to redefine it whereas static method it belongs to the class so you cannot override a static method right so it’s basically what we call it as in a class to class overriding which is known as hiding right so here the concept is known as hiding so if there is a static method in the parent class and if you want to create a method in the child class to override the static method it should also be static and that concept will be referred to as hiding now so what is multiple inheritance and does Java support it so guys multiple inheritance is one class extending more than one classes so this is not supported in Java so if you can see there is a diamond problem which comes into action that if there is a class A right and B and C class they are inheriting the a class and there is a Class D which is getting inherited from B and C so two copies of a will be coming into the last class so this kind of problem is not solved in the Java it needs the virtual Concepts to come into action since there are no virtual keyword or any other you know pointer support a in Java so we cannot solve this problem now right so Java multiple inheritance is not supported so one more query now over here right what will be the output of this program so if the parent is having this method F which is marked as static and one method bar which is a non-static method and there’s a parent child relationship where we are hiding a method versus overriding a method and you are executing these method calls right so guys the output will be so since if you can see over here so this parent. Fu and child. Fu then parent. barar and child. bar right so you’ll be able to get all these four outcomes I’m am Fu in super in child then child and then child right so just try to execute the code and get the behavior similarly we got one more program so guys this is where we got multiple Constructors and you created the object for the a so by default default Constructor will come into action but this over the reference function call Will execute the Constructor over here and this over here will execute the Constructor over here so you will get how are you then hello and then the height right so this as a Constructor execution will happen over here as a function call so you’ll get the output in the reverse order now right so one more program guys what’s going to be the output for the program so it’s basically one miscellaneous program where you got a block and a static block so guys a block is executed when you create the object of a class whereas static block is executed when the class is loaded into the memory so the idea is to Define you guys the role for the static block and a normal block so a block over here if you can see is executed when the object is constructed and static block is executed when the class is loaded into the memory so just try to run this program and come up with this output guys all right guys so now let’s understand seret right so this is where we’re going to pursue with the Enterprise Edition concepts of java so we have left out the core part of java and now we are moving into the advanced part of java so serx is also one of the Core Concepts when we talk of the Enterprise Edition of java so let us understand what exactly is a Ser right so that’s like what is a serlet so serlet is one Java program which is executed over the server now that’s like one very basic definition to what exactly is a server so what happens now whenever you’re writing an Enterprise application you need to have some code which is executed by the server so if the client is sending a request via HTTP or the https request will come to a server and the server will execute your Java program and it’s going to send back the response to the client now the client can specifically be browser or it can be a mobile application or any other kind of a client di right so Java serlet is a server site technology which will extend the capability of web servers by providing support for dynamic response and data persistence we got Java x. serlet and Java x.l. HTTP package which provides the various apis for writing the serlet program now any serlet program should implement the seret interface along with the life cycle methods where you can manage your cod snet as most web applications they are accessed using the HTTP protocol so we got a class known as HTTP serlet and we got a corresponding hierarchy for the same so over the web we use two of the major methods one is the get one is the post so what is the difference between get and post so guys get and post we got two major methods how we can send a request from a client to the server other than that we also have various other methods like put delete PR so on and so forth so these are the two important methods get is when you need to have limited amount of data which can be sent right because the data is sent as a part of URL now so post is when you have a large data and you need to send it across the body so it automatically comes up that the data is not visible as a part of URL and uh it is more secure in in terms of post request right so security is more because the data is not not exposed in the URL other than that you cannot can notot bookmark the post but the gets can be bookmarked we got item po versus non itm poent so guys what is item Port when you send a request you get a response you again send a request you again get a response and you compare the two responses whether they are equal or not right so it important so this guy get is more efficient right and it is more used than the post right where now it depends right so efficiency is like get is more efficient undoubtedly but it depends upon our requirement where you need to use what okay so let’s say you want to do a sign in so obviously we need to send the username and the password to the server and you cannot send it across the URL even though you can send it but you need to encrypt the data before you send it right so that’s like the differences between get and post now what is meant by the session management right so guys session is one term which means the time interval between the client and the server so we have techniques to manage the data within the session so that’s like the session management now so what are the various methods so number one we got the user authentication then the hidden form Fields Cookies URL rewriting and we got HTTP session API also to manage the session so session is a conversation State between the client and server and it can consist of multiple requests and responses between both of them now HTTP is a stateless protocol you cannot manage and maintain the state so that’s why we need to use some fundamentals through which we can come up with this right so guys every session will have a session ID which will be passed between the client and the server during the request and response and you can use various different techniques so to name few we have mentioned them right now we also have two apis known as serlet context versus serlet config so what is the difference over here so Ser context and config is where we need to extract the data which is stored somewhere as in part of your web.xml deployment descriptor so we will consider that part now so serlet config is a single object where serlet context is on a global level now right so it’s like local parameter associated with a particular serlet whereas we got a global parameter so both of them are the key value pairs right mentioned within the web.xml file whereas serlet config so you mention the data within the serlet whereas serlet context is mentioned out of the serlet tag in the serlet program you can use the API get serlet config and get serlet context and thereafter on the basis of the key you can retrieve the data right so for example shoing card of a user is specific to a particular user so you can use serlet config whereas to get the Mind type of a file or any kind of a global level or a a variable you can use the serlet context now what is a possible life cycle of a serlet now this is uh really important when you talk of servlets guys right so uh servlets they got five stages when you talk of the life cycle right the first of all the Ser will be loaded then it’s going to be in instantiated initialized then a request will be sent to a serlet and then finally the serlet will be destroyed so we got in it service and Destroy as the three life cycle call backs involved when you write the serlet program so you got three of the apis now init service and Destroy what is request dispatcher if you want to share the data from one Ser to the other Ser evenly right so we got this guide known as requ EST dispatcher which can forward the request to the other resource now this other resource can be possibly a serate or an HTML file or a GSP file and we can do so using the two different methods one is the forward method other one is the include method so what happens over here is when you send out the requests from the seret one to seret 2 when you use the forward method response generated from the ser let 2 will be sent back to the client in case of the include method response from the serlet 2 as well as the serlet one will be appended and then sent back as a final response to the client now we got request dispatcher coming up with the forward and include in these two differences so guys one of the important concept associated with the ser session management is cookies right so what exactly a cookie is so cookies are the textual data which gets stored on the client machine and this data is sent from the server so data is stored in the local machine of the client guys let us see this part right so if you can see over here I’ll show you this thing so when you open down the settings of your browsers right and you try to say cookie right so if you can see over here so you get this options of cookie coming up over here content setting you can see cookies so allow the local data to be set which is evenly recommended so guys cookies are the textual data which gets stored on your machine and you from the server can just use the method known as get cookies right to read out all the cookies and you can create a cookie by using the APA cookie itself it’s a hashmap key value pair cookie C1 assign new cookie with the key and the value right and if you want to add the cookie to the client’s machine you just call the method add cookie over the response object so the next part is the Java database connectivity so guys this deals with the database connections now right so Java database connectivity we got the first thing as jdbc driver so whenever you want to create a connection you must must have a driver right so guys driver is the layer which will help us to interract our jdbc apis with the database so there are four major types of drivers we call them type one type two type three and type 4 driver it goes like jdbc odbc Bridge driver native API driver which is a partial Java driver then if you want to come up with a client server model you got Network protocol driver and lastly you got got the pure kind of a driver thin driver the fully Java driver which will’ll be using to interact directly with the database so Java app it’s going to use the jdbc API and the jdbc driver manager will interact with the driver and drivers will further interact with the drivers for the databases libraries and so and hence this whole channel of communication will be established so what are the various steps to connect to the database in Java we use jdbc and there are some sequential steps which you need to perform so that we can perform all the crude operations so what are these now so there are five sequential steps which you need to perform so guys any of the database whichever you are using right so you need to come up with the documentation over their website right uh what is their driver class name what is the URL so on and so forth anyways let us have a look on to the snippet now so the first one is registering the driver which comes through the API known as class. forame so here you manage to write the driver type for driver or any other driver which you are going to use then you need to create the connection which is created with the help of an API known as driver manager so you say driver manager. getet connection we need to pass three things over here the first thing is the URL where your database resides so if it is on the Local Host you mention the host name to be the local host and you manage to write the port number and the database name then the username and the password for your database thereafter you need to say create statement over the connection once you have created the statement thereafter you’ll be able to execute any of your SQL query but in the finally block you must close your connections right so this is important that you manage to close your connections regardless or whether exception is coming or not coming so what are the different jdbc API components so guys we got various interfaces and classes coming into the Java jdbc package so we got java.sql package and many of the libraries if you can see the apis are mentioned over here so in the interface part we got connection statement prepared statement result set result set metadata database metadata and callable statement so all these are the interfaces so connection is used to have a connection statement to execute the SQL statements prepared statement is also used to execute the SQL statements but prepared statement is different from the statement in terms of how you are going to substitute the data into the SQL statement so here we use something known as wild card correctors as question mark right result set is the retrieved data from the database and the data about the retrieve data is known as result set metadata and the data about the database is known as database metadata callable statements are used so that you can execute the stored procedures of your databases and some of the few classes which are mentioned in the right hand side it includes driver manager which is used to get the connection so Bob and clob are the two apis where you can manage the binary large objects and the character large objects so binary large object means the audio files the video files right whereas clob means the textual files then the type will Define what are the various types SQL exception is exception in the jdbc if there is any connectivity issue or any other syntactical issue or any different kind of SQL issues which going to occur at one time so SQL exception is the class to get the error from now what exact a jdbc connection interface B right so guys the connection interface is the one which will maintain a session with the database so here we can uh use the transaction management logic which means Atomic City so Atomic City means that all and everything should happen at once so let us understand so connection is one important interface with the help of which we can manage the transaction for the transactions let us assume I want to transfer the amount from one account to the other account so there needs to be two possible SQL statements which are supposed to be f one will be the updation in my account that will be deduction of the payment in the other account again an updation query which will be the addition of the payment so these two queries should be fired at once and if any one query fails the other query should be rolled back so for the transaction management V are the connection you create the statement and thereafter you can come up with the autoc commit API where you can mark it as true and then you can use the apis like commit and roll back to manage your transactions so close is a method where you are able to close the session with the database right guys so there is one more possible difference between three of the apis that’s like execute execute query and execute update so guys execute query execute update and execute these are the three different apis all the three apis are used to perform the SQL operations over the database but query method is always focused to do a retrieve operation if you can see now select operation right so whereas your execute update method it’s going to perform the DML and the ddl operations insert update delete create an alter of the table right whereas execute method it is able to perform any kind of operation now so this is typically used to retrieve the data here the data can be modified whereas execute method any kind of SQL statement return type of these methods they are different now so result set will be returned from the execute query method here we’ll be getting some integer output and here we’ll be getting some Boolean output this method is used to execute only select queries so non- select queries and here both the select and non- select queries right guys so that’s like the jdbc part right so how we got various apis available in the Java database connectivity but we need to come up and understand there are the five different steps so you start from loading the driver then creating a connection writing the statements executing the statements and finally moving on to the closing of the statements so the next part is the framework of Frameworks that’s like the spring framework so if you know this if you have a hold on your spring framework guys so it’s like you are done with almost everything now right so one of the most important framework in the Enterprise Edition of java let us see what is spring framework so what exactly is spring guys so spring is

    an application framework which focuses on a concept known as inversion of control for the Java platform so many of the Great Technologies they are built looking onto the architectural model of the spring framework even the Android how the UI part is linked with the layout part so that concept is again adopted from the spring pattern right so it is one of the pattern which focuses on ioc that’s like inversion of control so the framework core features they can be used by any Java app app for building an Enterprise Edition application one of the major advantage of writing a spring framework app is it’s lightweight it is having an integrated framework architecture where we can use it to develop any kind of an Enterprise app in Java right so it’s less on a memory it’s high on performance and it contains various Frameworks available within it through which you can write a multi-purpose Enterprise application so guys what and all are the modules available in the spring framework now so spring framework has various modules to start with we got the very first layer as the core container if you can see this core layer so whole of your spring framework is dependent on this core layer so this core layer focuses on something known as inversion of control and dependency injection so we got this as spring context dependency injection then aop is aspect oriented programming which focuses on how we can fulfill crosscutting concerns then the database module of the spring framework includes Dow jdbc and orm as in the design patterns so Dow the data access object so here we can write the Dow patterns to access the database we got jdbc support we also got the hibernate or the OM support which is object relational mapping the web module of the spring framework can help us to write the web applications where spring MVC is one of the majorly used Frameworks in the market now so model view controller where model represents the data container view represents the UI or the presentation and controller represents the business logic as in seret now right so so spring MVC is one of the very important concern when it comes to develop the web applications and we got various web services coming out in this module itself let us move on to some annotations which are used in the spring configuration so guys there are various annotations now which are used in the spring framework now at theate required right so it’s like what is the input which is required Auto wiring for your dependency injection what happens pre- destroy what happens post construct right so before the object is finalized just after the object is constructed what is the resource what is the qualifier so these are some of the spring annotations which are used for the configuration purpose so what is Bean in Spring and what can be the different scopes of The Bean in the spring right so guys I’ll take up one very uh a basic example over here to mention this concept right so let us take one example so I got this spring demo available with me with all the jar files available out here we got this employe bean. XML file so just give me a moment right so all right so here you can see now we got this employee beam so this is one XML configuration for the spring framework so we got cod. ed. employee as a class right and here what I able to do is to define the property value tags for my B right in the employee. Java I can say it’s basically a bean or a pojo so what is a bean class Bean class is a class which will have the attributes along with the setter and the Getters right so all the setter get methods are also there in my client class what I’m going to do I’m just going to use this spring container and thereafter I will just extract my employee object right so just let me do this part all right so I got the reference of the employee bean. XML and in my class I got this mban one so I’ll just say m being one and thereafter I’ll do a ceso of M right so let’s try to run the code and you can see that the data over here is coming up as an ID name and salary right I have just over writed this two string method out here so along with this I’m also supposed to do a super do two string right I’ll do a back sln and a super do two string so now it will show me the data within the object as well as the reference variable m is pointing to the address it is showing me the address as well to the object now so the bean is extracted from the spring container right so there are various Scopes available now for example same MB one in my client if I try to get it again right so let us try to say get of bean again right as in M1 and M2 let us say this is M2 and this is M1 right so I will closely observe one difference I mean one similarity that every time when you request this guy get Bean for the mban one you are able to receive the reference to the same object scopes of the beam can be mentioned over here using attribute known as scope and here now I’m going to mention it as prototype you can see we got this guy as a prototype now when I run my code again you’ll find one significant difference that here the reference is coming out to be different it means now by default it was giving me the singl ton as a design pattern but when I change the scope to prototype every time I am requesting the object of M being one from the spring container it is now constructing a new object and returning it back to me so guys this is what we got as in what can be the scope of your being in the spring so inversion of control container can have various Scopes now single ton means only single object which is by default so prototype means you can have every time a new object request means one object per entire request session means one object per entire session and Global session means one object per entire Global session on an application Level so how is a being added to a spring application so guys you use this tag known as be you mention the ID and then you can have the properties as in key value pair right so what is the role of dispatcher serlet and context loader listener so guys dispatcher seret is acting as a front controller in the spring MVC so many times I will call it as a receptionist now so what happens any request when it’s going to land up to your web application so this guy dispatcher serlet will dispatch the request to the corresponding resource available on your web application so it’s it’s going to act as a receptionist in your program now whereas context load a listener it’s a listener which will give you the event call backs when the context will be loaded it’s like startup and shutdown of the web application context in the spring route now what is the difference between Constructor in injection and Setter injection so before you come up with the Constructor injection and the setter injection you must understand what is dependency injection so guys uh let us understand this flow once again I’m having one class as employe right and there is one more class known as address so employee can have the name employee can have some salary there will be definitely an employee ID and employee can also have an address right so I’ll just say an address to be as adrs so what we can do we can create one Constructor of an employee in this way right and here you can say Eid goes like 101 name goes like let’s say John salary goes like 30,000 and address goes like some new address right where address can have some address line then you can have some City you can have a state and lastly we can also have some kind of a zip code just mentioning some parameters over here now what will happen over the client part you will be writing employeee assign new employee so whenever you you are creating the object of employee this becomes the concept that the Constructor of the employe will create the object of the address we can see a significant dependency now when you are creating the object of employee within the Constructor of employee the object of address will be constructed we say this concept as high dependency or you can say we got a higher value of coupling now we need to reduce it right so you need to bring it to the lower level so you need to take it as a low coupling so what I will do I will not use this technique I will say let us create one more Constructor and pass the address as an input right and thereafter you can say address assign a or you can create a Setter you can say set address and you can say address and here you pass this guy so guys in these two approaches rather than having a single employee object you can now create an employee object you can create address objects separately right you need not to be dependent upon the employee object and later you can pass the address object or you can say employee e assign new employee and E dot set the address and you can pass this guy a so we want to reduce the dependency or we want to come down with the coupling parameter we created this logic out here so Setter injection and Constructor injection are the two terminologies linked up in order to justify these two terms now right so let us open up our presentation once again and look on to Constructor injection versus seter injection so there is no partial injection so seter injection can lead to partial injection here we need not to override the setter property but here the Constructor property in case both the things are defined so you can create new instance if any modification occurs so you need not to create any instance if there is a property change so better for too many properties better for few properties so guys depends upon your choice but Constructor and seter injection they are the two key Concepts to bring down your dependency to the lower levels now what is auto wiring in spring so many of times when we are writing this dependency injection programs you can perform these injection programs based on the names right for example you can mention this auto wirring tag and you can say by name so you need not to now write an explicit injection logic it will be mapped automatically for you so there are various modes through which you can have an auto wirring coming into action so no is the default mode it means you don’t specify the autowire it means here uh nothing is enabled so Auto wiring is not enabl so when you mention by name it means you take the seter method and you map the data corresponding to the name by type will use the type of the attribute and Constructor will inject on the basis of the Constructor so how can we handle any exceptions in Spring MBC framework right guys so there can be exceptions now when you are writing the program right so spring MVC framework can help us to achieve robust programming concept through these possible ways Number One controller based so you can define an exception hander method in your controller class itself you can have a global exception hander where exception handling is is a cross cutting concern to aop right and then you can have a Handler exception resolver so any spring Bean which you have declared in the dispatcher serlet application context that can Implement Handler exception resolver which can thereafter be used to intercept and process any exception which occurs in the MVC system and it is not handled by the controller part so you can use the various different concepts out here if you can see add the rate exception hander annotation so that you can specify that this view will be used to display some arror all right so some important annotations again in the spring framework guys we got at theate controller at theate path variable qualifi configuration scope request mapping response body auto wired service and and aspect right so all these annotations they are quite important Concepts and you must have a fair knowledge on all these right for example at theate Auto so either you can configure it in XML file or you can use annotations so XML files they come up with the inversion of control concept whereas annotations you need not to have XML files you can use them directly all right guys so lastly in the spring framework we can also have spring spring framework coming up with the database connectivity either through jdbc or through hibernate or any other framework now right so spring and hibernate they can work together we have some om modules available within the spring framework and we got an API known as hibernate template so if you want to come up with the spring and jdbc you got jdbc template API right so guys spring omm module will help us to integrate hibernate framework into a spring application and spring orm also provides the support so that you can use spring declarative transaction management you can use annotations and you can perform the actions now so now we will be on our last concept that’s like the hibernate which is the backend framework based on object relationship mapping right so guys let us see what and all we got in the hibernate now so the 44th question is what is hibernate hibernate is an OM which is object relationship mapping it’s a framework now where you just mention your Java object and you forget about your database tables so guys what we going to do we’re going to write down some mapping files as an XML configuration files where you will be Mapp that this attribute in my object in my class will be the column name in the respective relational table so hibernate provides a reference implementation of the jpa Java persistence API right and it makes a great choice as an OM tool where we got the benefits of lose coupling now I hope you got the meaning of loose coupling where dependency has been reduced to the maximum level so hibernate configures all the flexible and can be done from the XML configurations as well as through annotations programmatically right so it it provides us both the ways now so what can be the benefits with the hibernate framework so guys hibernate will eliminate all the boilerplate code whereas in the traditional approach you need to mention and write various code Snippets and you need to write the relationship mappings the cache management everything is uh you know eliminated when you use hybrid now so this framework will provides us uh XML support as in through configuration files and annotational support through jpa that’s like Java persistent apis now hibernate provides a very powerful uh language that’s like hql which is very much in line to SQL but you need not to learn it it’s more simpler than the SQL even now SQL itself is very simple language you got more simpler language than the SQL that’s like hql so guys it’s an open-source project from the red hat community and uh it is used worldwide and it’s very famous especially for any large Enterprise application your hibernate framework is one of the ideal choices now now any database vendor specific feature hibernate is suitable because we can also execute native SQL queries so hybern is not just that we can use its native apis and you can’t do anything out of the box you got to execute the SQL queries also with the hibernate one of the most significant features of the hibernate is the cach management so hibernate provides three levels of cash through which your performance of the database operations they’re going to get improved now so what is the architecture of hibernate guys so hibernate of having an architecture in a very uh simpler fashion guys we got a database on top of database we got various you know uh modules coming in for example Java transaction apis we got Java database connectivity and thereafter we got j& di that’s like Java naming and directory interfaces on top of it we got the hibernate apis so configuration API will read the XML configuration file and it will help us to create a session Factory session Factory will be used to create the session and session is one API which maintains and creates a connection with the database using the transaction API you can use the commit and roll back features and query API will help you to create one query and retrieve the data criteria is one API which will help you to mention the criteria for example you got restrictions to you know manage for example greater than less than all equal to so all these are criterias when you are trying to retrieve the data in the Java application you just write a pojo class you just write a bean and you give this object to the hibernate and hibernate API they’re going to manage everything for you all right we got the get method and the load method in order to to perform the retrieve operation so let’s see the differences let’s say we we don’t have a record available in the database so load method will show an exception that’s like object not found exception whereas the get method will return a null as in the object is not found so guys get method will always hit the database whereas load method will not hit the database right in case it’s not there and you’re going to get get a real object not a proxy object whereas load will return you a proxy object so get should be used if you are not sure about the existence of instance now so let’s say that the record is existing or not I don’t know so you use get if you are sure about whatever you are going to extract from the database it is existing you can use the load method so there are now significant differences between hibernate and jdbc guys this is one of um the major query between you know as in the part of interviews so why hibernate and why not jdbc what are the differences what are the advantages that you should be opting for hibernate now if you’re writing a very small app right you need not to go on the hibernate level hibernate level is required when you’re writing an Enterprise app right where you need to save the time and you don’t want to do a rework and there are various other differences so let’s have a look onto that now so guys first of all you need to write lot of code when it comes to jdbc whereas hibernate you have very limited instructions to mention right hnet supports all the association inheritance collection mappings where in case of jdbc APA you need to write all those Logics so one to one one to many many to many all these mappings they are available as a support in the hibernate hibernate will also give you the transaction management implicitly whereas in jdbc again you need to manage that part Now jdbc API they will throw SQL exception right so it’s a checked exception so we need to write lot of TR catch block code when you are using this uh jdbc but hibernate will internally manage this uh TR catch block for you we got hql which is objectoriented programming whereas jdbc you need to write the native SQL queries hibernate will automatically give you the cash management for better performance in three levels of cash whereas jdbc you need to manage it yourself so by default hibernate’s performance is higher as compared to the jdbc part so lastly hibernate will provide an option where we can create the database and the tables too but for the jdbc these EX existence should be there before you dump the data other than that one of the very significant feature why hybernate over jdbc is if you want to switch from one database to the other database it’s going to be very complicated with the jdbc but in case of hibernate we got a configuration file you just need to change the parameters in the configuration file and you will be able to shift from Oracle to my SQL or any other database to any other database of your choice and with this we come to an end to this Java full course if you enjoyed listening to this full course please be kind enough to like it and you can comment on any of your doubts and queries we will reply to them at the earliest and do look up for more videos and playlist And subscribe to the edas YouTube channel to learn more thank you for watching and happy learning

    By Amjad Izhar
    Contact: amjad.izhar@gmail.com
    https://amjadizhar.blog