SICT

WEB700

Web Programming Foundations

Schedule Notes Resources Graded Work MyApps Instructions Vercel Guide Code examples

WEB700 Week 1 Notes

Preface

The web is the most ubiquitous computing platform in the world. As a developer, learning the web takes time. There are hundreds of languages, libraries, frameworks, and tools to be learned, some old, some built yesterday, and all being mixed together at once.

The fundamental unit of the web is the hyperlink–the web is interconnected. These weekly notes provide numerous links to external resources, books, blogs, and sample code. To get good at the web, you need to be curious and you need to go exploring, you need to try things.

Make sure you follow the links below as you read, and begin to create your own web of knowledge and experience. No one resource can begin to cover the breadth and depth of web development.

Question: do I need to read the weekly notes? How about all the many links to external resources?

Yes, you do need to read the weekly notes. You will be tested on this material. We will discuss it in class, but not cover everything. The external links will help you understand and master the material. You are advised to read some external material, but you don’t need to read all of it.


Internet Architecture

Overview


Application Protocols

The web runs on-top of TCP/IP networks using a number of communication protocols, including:

There are many more as well (SMTP, FTP, POP, IMAP, SSH, etc).

We often use the terms “Web” and “Internet” interchangeably, however, they aren’t the same.

The World Wide Web (WWW) runs on top of the Internet using HTTP, and allows us to access web services, request resources (i.e., pages, images), and transmit data between clients and servers. The web is a subset of the Internet.

The web isn’t owned or controlled by any single company, organization, or government. Instead, it is defined as a set of open standards, which everyone building and using the web relies upon. Some examples of these standards include HTML, HTTP, SVG, and many more.


HTTP Requests and Responses

The Hypertext Transfer Protocol is a stateless, client-server model for formatting requests and responses between computers on the Internet. This means one computer makes a request (the client) to another (the server), and after the response is returned, the connection is closed.

The server listens for requests, and fulfills (or rejects) those requests by returning (or generating) the requested resources, such as images, web pages, videos, or other data.

URLs

Web resources are reachable via unique identifiers called a Uniform Resource Locator or URL. Consider the URL for the official Node.js documentation (A JavaScript runtime that we will use throughout this course):

https://nodejs.org/en/docs/

A URL contains all the information necessary for a web client (e.g., a browser) to request the resource. In the URL given above we have:

URLs can only contain a limited set of characters, and anything outside that set has to be encoded. This includes things like spaces, non-ASCII characters, Unicode, etc.

Requests

A URL describes the location (i.e., server, pathname) and how to interpret (i.e., which protocol) a resource on the Internet. To get the resource, we need to request it by sending a properly formatted HTTP Request to the appropriate server (host):

GET /en/docs HTTP/2 
Host: nodejs.org

Here we do a GET request using HTTP version 2 for the resource at the path /en/docs on the server named nodejs.org.

There are various HTTP Verbs we can use other than GET, which allow us to request that resources be returned, created, deleted, updated, etc. The most common include:

We can use a URL in many ways, for example, via the command line using a tool like curl:

$ curl https://nodejs.org/en/docs/

<!DOCTYPE html>
<html lang="en" >
...
</html>

Responses

Upon receiving a request for a URL, the server will respond with an HTTP Response, which includes information about the response, and possibly the resource being requested. Let’s use curl again, but this time ask that it --include the response headers:

$ curl --include https://nodejs.org/en/docs/

HTTP/2 200 
date: Fri, 26 Jul 2019 18:57:35 GMT
content-type: text/html
set-cookie: __cfduid=da9eda0bdcd9e97e879074e4b64bfcc221564167455; expires=Sat, 25-Jul-20 18:57:35 GMT; path=/; domain=.nodejs.org; HttpOnly
last-modified: Tue, 23 Jul 2019 21:05:15 GMT
cf-cache-status: HIT
age: 46
expires: Fri, 26 Jul 2019 22:57:35 GMT
cache-control: public, max-age=14400
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 4fc899a5ae3ee1ce-ORD

<!DOCTYPE html>
<html lang="en" >
...
</html>

In this case, we see a two-part structure: first a set of Response Headers; then the Response Body formatted using HTML (Discussed in detail in week 6). The two are separated by a blank line. The headers provide extra metadata about the response, the resource being returned, the server, etc.

HTTP Headers are well defined, and easy to lookup via Google, MDN, or StackOverflow. They follow the key: value format, and can be upper- or lower-case:

name: value

For example:

content-type: text/html, where content-type is the name and text/html is the value.

In the response above, we see a number of interesting things:

After these headers we have a blank line (i.e., \n\n), followed by the body of our response: the actual HTML document.

What if we requested a URL that we know doesn’t exist?

$ curl --include https://ict.senecacollege.ca/course/web000

HTTP/1.1 302 Found
Date: Thu, 30 Aug 2018 20:25:28 GMT
Server: Apache/2.4.29 (Unix) OpenSSL/1.0.2l PHP/5.6.30
X-Powered-By: PHP/5.6.30
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0
Location: https://ict.senecacollege.ca/Course/CourseNotFound?=web000
Content-Length: 0
Content-Type: text/html; charset=UTF-8

This time, instead of a 200 status code, we get 302. This indicates that the resource has moved, and later in the headers we are given a new Location to try. Notice there is no body (not every response will include one).

Let’s try following the suggested redirect URL:

$ curl --include curl --include https://ict.senecacollege.ca/Course/CourseNotFound?=web000

HTTP/1.1 404 Not Found
Date: Tue, 27 Aug 2019 13:08:55 GMT
Server: Apache/2.4.6 (CentOS) mpm-itk/2.4.7-04 OpenSSL/1.0.2k-fips mod_fcgid/2.3.9 PHP/5.6.40
X-Powered-By: PHP/5.6.40
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0
Content-Language: en
Link: </?q=Course/CourseNotFound>; rel="canonical",</?q=node/891>; rel="shortlink"
X-Generator: Drupal 7 (http://drupal.org)
Transfer-Encoding: chunked
Content-Type: text/html; charset=utf-8

...

Now a third response code has been returned, 404 Not Found as well as another HTML page telling us our course couldn’t be located.

There are dozens of response codes, but they fall into a few categories you should learn:


Web Browsers

So far we’ve been communicating with web servers using curl, but a more common tool is a Web Browser.

A good way to think about a browser is as an operating system vs. an application. A web browser provides implementations of the web’s open standards. This means it knows how to communicate HTTP, DNS and other protocols over the network in order to request resources via URLs. It also contains parsers for the web’s programming languages, and knows how to render, execute, and lay-out web content for use by a user. Browsers also contain lots of security features, and allow users to download and run untrusted code (i.e., code from a random server on the Internet), without fear of infecting their computers.

Some of the the largest software companies and vendors in the world all have their own browsers:

There are hundreds more, and thousands of different OS and version combinations. There are good stats on usage info for desktop and mobile, but no one company or browser controls the entire web.

As a web developer, you can’t ever know for sure which browser your users will have. This means you have to test your web applications in different browsers and on different platforms in order to make sure the experience is good for as many people as possible.

The web is also constantly evolving, as new standards are written, APIs and features added to the web platform, and older technologies retired. A good way to stay on top of what does and doesn’t work in a particular browser is to use https://caniuse.com/. This is a service that keeps track of web platform features, and which browsers do and don’t implement it.

For example, you can look at the URL() API, used to work with URLs in JavaScript: https://caniuse.com/#feat=url. Notice that it’s widely supported (green) in most browsers (89.69% at the time of writing), but not supported (red) in some older browsers like Internet Explorer.

Because the web is so big, so complicated, so old, and used by so many people for so many different and competing things, it’s common for things to break, for there to be bugs, and for you to have to adapt your code to work in interesting ways. The good news is, it means there are lots of jobs for web developers to make sure it all keeps working.


Uniqueness of the Web as a Platform

We’ve been discussing HTTP as a way to request URLs be transferred between clients and servers. The web is globally distributed set of

The web can be read-only. The web can also be interactive (video games), editable (wikis), personal (blog), and productive (e-commerce).

The web is linkable, which makes it something that can be indexed, searched, navigated, and connected. The web gets more valuable as its connections grow: just look at all the other pages and resources this page links to itself!

The web allows users to access and run remote applications without needing to install new software. The deployment model of the web is HTTP. Compare that to traditional software that has to be manually installed on every computer that needs to run it. Same with mobile phones and apps in the various app stores. Updates get installed every time you use a URL.

Question: how many mobile or desktop apps did you install today vs. how many websites did you visit?

The web works on every computing platform. You can access and use the web on desktop and mobile computers, on TVs and smartwatches, on Windows and Mac, in e-Readers and video game consoles. The web works everywhere, and learning how to develop software for the web extends your reach into all those platforms.


Introduction to JavaScript

The first web technology we will learn is JavaScript. JavaScript (often shortened to JS) is a lightweight, interpreted or JIT (i.e., Just In Time) compiled language.

JavaScript looks similar to C/C++ or Java in some of its syntax, but is quite different in philosophy; it is more closely related to Scheme than C. For example, JavaScript is a dynamic scripting language supporting multiple programming styles, from object-oriented to imperative to functional.

JavaScript is one of, if not the most popular programming languages in the world, and has been for many years. Learning JavaScript well will be a tremendous asset to any software developer, since so much of the software we use is built using JS.

JavaScript’s many versions: JavaScript is an evolving language, and you’ll hear it referred to by a number of names, including: ECMAScript (or ES), ES5, ES6, ES2015, ES2017, etc. ECMA is the European Computer Manufacturers Association, which is the standards body responsible for the JS language. As the standard evolves, the specification goes through different versions, adding or changing features and syntax. In this course we will primarily focus on ECMAScript 5 (ES5), which all browsers support. We will also sometimes use newer features of the language from ECMAScript 6 (ES6), which most browsers support. Language feature support across browsers is maintained in this table.


JavaScript Resources

Throughout the coming weeks, we’ll make use of a number of important online resources. They are listed here so you can make yourself aware of them, and begin to explore on your own. All programmers, no matter how experienced, have to return to these documents on a routine basis, so it’s good to know about them.


JavaScript Environments

Unlike C, which is compiled to machine code, JavaScript is meant to be run within a host environment. There are many possible environments, but we will focus primairly on the following:

If you haven’t done so already, you should install Node.js

JavaScript Engines

JavaScript is parsed, executed, and managed (i.e., memory, garbage collection, etc) by an engine written in C/C++. There are a number of JavaScript engines available, the most common of which are:

These engines, much like car engines, are meant to be used within a larger context. We will encounter them indirectly via web browsers and in Node.js.

It’s not important to understand a lot about each of these engines at this point, other than to be aware that each has its own implementation of the ECMAScript standards, its own performance characteristics (i.e., some are faster at certain things), as well as its own set of bugs.


Running JavaScript Programs

JavaScript statements can be stored in an external file with a .js file extension, or embedded within HTML code via the HTML <script> element. For the first part of this course, we will use Node.js to execute our JavaScript code from the command line within our development environment, Visual Studio Code

  1. If you have not yet done so, download and install Visual Studio Code and Node.js

  2. Create a folder somewhere on your local machine and give it a relavent name, ie: “Ex1” (Note Before starting any coding project in Visual Studio Code, it’s important to first create a containing folder).

  3. Open Visual Studio Code

  4. You should see a large, “Open Folder” button in the left pane. If the left pane is not visible click the top-left icon (it should look like two files overlapping)

  5. Click the “Open Folder” button and choose your newly-created “Ex1” folder.

  6. Next, hover your mouse over the “EX1” text in the left pane; this will cause 4 icons to appear. Click the first one (It looks like a little file with a plus (+) icon) to create a new file within the “Ex1” folder.

  7. Name this file “Hello.js” and press enter. Once this is complete, “Hello.js” should be open and ready for editing.

  8. Enter the following line of code:

    console.log("Hello World!");

  9. Save the file and open the “Integrated Terminal” using either the “View” menu and choosing “Terminal” or using the key combination ctrl + `

  10. You should now see a terminal / Command Prompt at the bottom of your code. Within this terminal, execute the following command:

    node Hello.js

  11. “Hello World!” - you have just executed your first line of JavaScript code!


JavaScript Syntax

Important Ideas

JavaScript Variables

Variables must start with a letter (a-zA-z), underscore (_), or dollar sign ($). They cannot be a reserved (key) word. Subsequent characters can be letters, numbers, underscores.

NOTE: If you forget to use the var keyword, JavaScript will still allow you to use a variable, and simply create a global variable. We often refer to this as “leaking a global,” and it should always be avoided:

var a = 6;      // GOOD: a is declared with var
b = 7;          // BAD: b is used without declaration, and is now a global


JavaScript Operators

Common JavaScript Operators (there are more, but these are a good start):

Operator Operation Example
+ Addition of Numbers 3 + 4
+ Concatenation of Strings "Hello " + "World"
- Subtraction of Numbers x - y
* Multiplication of Numbers 3 * n
/ Division of Numbers 2 / 4
% Modulo 7 % 3 (gives 1 remainder)
++ Post/Pre Increment x++, ++x
-- Post/Pre Decrement x--, --x
= Assignment a = 6
+= Assignment with addition a += 7 same as a = a + 7. Can be used to join Strings too
-= Assignment with subtraction a -= 7 same as a = a - 7
*= Assignment with multiplication a *= 7 same as a = a * 7
/= Assignment with division a /= 7 same as a = a / 7
&& Logical AND if(x > 3 && x < 10) both must be true
() Call/Create () invokes a function, f() means invoke/call function stored in variable f
|| Logical OR if(x === 3 || x === 10) only one must be true
| Bitwise OR 3.1345|0 gives 3 as an integer
! Logical NOT if(!(x === 2)) negates an expression
== Equal 1 == 1 but also 1 == "1" due to type coercion
=== Strict Equal 1 === 1 but 1 === "1" is not true due to types. Prefer ===
!= Not Equal 1 != 2, with type coercion
!== Strict Not Equal 1 !== "1". Prefer !==
> Greater Than 7 > 3
>= Greater Than Or Equal 7 >=7 and 7 >= 3
< Less Than 3 < 10
<= Less Than Or Equal 3 < 10 and 3 <=3
typeof Type Of typeof "Hello" gives 'string', typeof 6 gives 'number'
cond ? a : b Ternary status = (age >= 18) ? 'adult' : 'minor';


JavaScript Expressions

A JavaScript expression is any code (e.g., literals, variables, operators, and expressions) that evaluates to a single value. The value may be a Number, String, an Object, or a logical value.

var a = 10 /2;                        // arithmetic expression
var b = !(10 / 2);                    // logical expression evaluates to false
var c = "10 " + "/" + " 2";           // string, evaluates to "10 / 2"
var f = function() { return 10 / 2;}; // function expression, f can now be called via the () operator
var d = f();                          // f() evaluates to 10/2, or the Number 5


JavaScript Execution Flow

JavaScript execution flow is determined using the following four (4) basic control structures:

/**
 * 1. Sequence example: each statement is executed one after the other
 **/
var a = 3;
var b = 6;
var c = a + b;


/**
 * 2. Conditional examples: a decision is made based on the evaluation of an expression,
 * and a code path (or branch) taken.
 **/
var grade;
var mark = 86;

if (mark >= 90) {
    grade = 'A+';
} else if (mark >= 80) {
    grade = 'A';
} else if (mark >= 70) {
    grade = 'B';
} else if (mark >= 60) {
    grade = 'C';
} else if (mark >= 50) {
    grade = 'D';
} else { 
    grade='F';
}

switch(grade) {
    case 'A+':
        // do these steps if grade is A+
        break;
    case 'A':
        // do these steps if grade is A
        break;
    case 'B':
        // do these steps if grade is B
        break;
    case 'C':
        // do these steps if grade is C
        break;
    case 'D':
        // do these steps if grade is D
        break;
    default:
        // do these steps in any other case
        break;
}


/**
 * 3. Looping example: a set of statements are repeated
 **/

var total = 0;
for(var i = 1; i < 11; i++) {
    total += i;
    console.log("i", i, "total", total);
}


/**
 * 4. Transfer example: a set of statements are repeated
 **/

function add(a, b) {        // declaring the add function
    if(!b) {                // check if the b argument exists/has a value
        return a;           // if not, simply return the value of argument a
    }
    return a + b;           // otherwise, return the two arguments added together
}

var total;
total = add(56);            // invoking the add function with a single argument
total = add(total, 92);     // invoking the add function with two arguments