Table of Contents

• Welcome
  • Indices and tables
• Principal Component Analysis (PCA)
  • Step 1) PCA
  • Step 2) Sphering
  • Step 3) Sphere-R
• Decision Tree and Ensemble Learning
  • Decision Tree
    • Order of decision; why Outlook is the first question/decision?
    • Entropy of the entire system
    • Information Gain
      • Outlook decision
      • Temperature Decision
      • Wind Decision
      • Humidity Decision
  • Titanic Example
    • Loading data using BeautifulSoup and urlib3 as the htm client
    • Processing data
      • Categorize passengers and crews
      • Check passenger ticket class
      • Check Adult/Child
      • Check Gender
    • Check the importance of each feature
    • Information gain analysis
    • Pick features and lable to construct the decision tree
    • Process categorical variables to numerical
    • Split to train and test data
    • Build a decision tree
    • Tree visualization
    • Evaluate the tree
      • Using train and test data sets
      • K-fold cross validation
  • Ensemble learning to avoid over fitting
    • Random Forest
    • Gradient Boosted Trees
    • Hyper Parameter Tuning
• Statistical Analysis for Income Data
  • Load the data
    • Check the Member_IDs corresponding to median values
    • Check Spatial Data
  • Extract Median Data
    • Choose a Subset
      • Data quality check
    • Alternatives for histogram plot
      • Using pandas built-in plot
      • Using matplotlib
      • Using Bokeh
      • Custom Plots
    • Location Map
  • Consider writing a wrapper (repeatability)
• National Water Data: HYDAT
  • Hydat data
    • Input Query (data Extraction)
    • Choose a subset of data
    • Break table (Days, Symbols)
    • Unpivot Days (melt)
    • Unpivot Symbols
    • Join tables
    • Drop/rename columns
    • Adding date
    • Export to CSV
    • Write to SQL server
      • Initialize a connection
      • Get server version
      • Create a new table
      • Write to SQL Server
      • Read Back From Server
    • Write to sqlite database
• Simple Implementation of Dijkstra Algorithm
  • Dijkstra algorithm
  • Generating random nodes and edges (an example network)
  • Initialize the algorithm solver
    • Set Nodes
    • Set Edges
    • Consider node A as the origin node
    • Path from origin node to any other node in the network
    • Plot shortest path from the origin node (i.e. A) to any other node in the network
• Summary Statistics for Well Data
  • vtk visulization for a petroleum reservoir model and sampled wells
  • Boundary modelling Data
  • Rock type and Petrophysical Properties
    • Get the global proportions for rock types
    • Uni/Bi variate relationships for different properties within each rock type
      • RockType 1
      • RockType 2
      • RockType 3
      • RockType 4
      • RockType 5
• Reshaping in Pandas - Pivot, Pivot-Table, Stack and Unstack
  • Pivot
  • Pivoting By Multiple Columns
  • Common Mistake in Pivoting
  • Pivot Table
  • Stack/Unstack
    • Stack