We should normalize or standardize data before applying KNN because the algorithm is distance-based, and unscaled features can distort distance calculations, leading to biased results. In this example, we’ll use the Iris dataset, which is… KNN classification: practical notices & implementation using Python & R
K-Nearest Neighbors (KNN) is a popular algorithm used for both classification and regression tasks. In KNN, the output is a class membership, which is assigned based on the majority of the k nearest data points.… K-Nearest Neighbors (KNN): an introduction
Linear Discriminant Analysis (LDA) is a classifier that creates a linear decision boundary by fitting class-conditional densities to the data and applying Bayes’ rule. The model assumes that each class follows a Gaussian distribution with… Linear Discriminant Analysis Implementation in Python & R
Stepwise feature selection is a systematic approach to identifying the most relevant features for a predictive model by combining both forward and backward selection techniques. The process begins with either an empty model. Then, we… Stepwise Feature Selection +example
Backward feature selection involves iteratively removing the least significant feature from a model based on adjusted R-squared. In this example, we are predicting nuts collected by squirrels, features like temperature and rainfall are chosen as significant predictors through this method. The process aims to finalize a model with the most influential features.
Forward feature selection starts with an empty model and adds features one by one. At each step, the feature that improves the model performance the most is added to the model. The process continues until… Forward feature selection: a step by step example
ElasticNet regression is a regularized regression method that linearly combines both L1 and L2 penalties of the Lasso and Ridge methods. This allows it to perform both feature selection (like Lasso) and maintain some of… ElasticNet Regression: Method & Codes
Motivation Now, recall that for LASSO Ridge Regression: Ridge regression: Ridge adds the penalty, which is the sum of the squares of the coefficients, to the loss function in linear regression. Ridge regression shrinks the… Ridge regression: method & R codes
The Lasso (Least Absolute Shrinkage and Selection Operator) is a regression technique that enhances prediction accuracy and interpretability by applying L1 regularization to shrink coefficients. Unlike traditional regression methods, Lasso forces some coefficients to become… Lasso Regression and LassoCV: methods & Python codes
An example of performing simple linear regression using train-test split where the process is as follows, 1. Generate a synthetic dataset: 2. Split the dataset: We use train_test_split to divide the data into training and… Simple linear regression using train-test split in Python & R
Reviewers are not required to read supplementary materials, but many do. Therefore, making your proof easy to read is important. General Guidelines: Example: Theorem 1: Let and be real numbers. If , then . The… How to Write a Proof in a paper
Dosage Optimization: Pharmacologists use derivatives to find the optimal drug dosage that maximizes therapeutic effects while minimizing side effects. The concentration of the drug in the bloodstream is modeled as a function of time, and… Example of using derivatives to find optimal drug dosage
Optimization of Material Usage: Engineers use derivatives to minimize the cost of materials while maintaining structural integrity. For example, determining the optimal dimensions of a container to minimize surface area for a given volume. Example:… Example of using derivatives to optimize the material cost
Problem 2: Find the minimum distance from the point (2, 3) to the line . Solution: More examples:
Detailed information can be found in Combining datasets to improve model fitting or its presentation slide. Summary: The key points of the paper titled “Combining Datasets to Improve Model Fitting” are as follows: Problem and… Combining datasets to increase sample size
Supplementary contents for section “Calculus can hurt” in the KSML app: Limit: Derivatives Integral
Expectation Maximization (EM) is an iterative algorithm used for finding maximum likelihood estimates of parameters in statistical models, particularly when the model involves latent variables (variables that are not directly observed). The algorithm is commonly… Expectation Maximization (EM) & implementation
Handling noisy data is a crucial step in data preprocessing and analysis. In general, here are some common approaches to manage noisy data: 1. Data Cleaning 2. Data Transformation 3. Statistical Techniques 4. Machine Learning… A comic guide to denoising noisy data
After you introduce the motivation of your work, it’s time to write a paragraph detailing the contribution of your work for a paper. In that paragraph, consider the following structure and key elements: Significance and… Writing the paragraph detailing the contribution of your paper
Recall that Missing Not At Random (MNAR) is a type of missing data mechanism where the probability of missingness is related to the unobserved data itself. Here are some more examples of MNAR: In each… A comical guide to Missing Not At Random (MNAR)
Missing at Random (MAR) is a statistical term indicating that the likelihood of data being missing is related to some of the observed data but not to the missing data itself. This means that the… What’s Missing at Random (MAR)?
Coding and managing projects Using AI to better code, debug and manage projects How to export an R dataframe to LaTeX Paper writing: Analyze experiment results faster with ChatGPT How to write in Latex faster… Tips & Tricks for research newbies
Plagiarism is the act of using someone else’s work, ideas, words, or intellectual property without proper acknowledgment or permission, and presenting it as your own. Even reusing your own previously published work or parts of… Check Plagiarism by grammarly
Multiple regression analysis can be used to understand the relationship between the waiting time to log in to Windows (dependent variable) and several independent variables. Let’s assume we have the following independent variables: Suppose that… Multiple regression analysis: waiting time to log in to Windows
1. Code Completion and Suggestions AI-powered code completion tools can predict and suggest the next lines of code based on the context of your current coding. Examples include: Copilot: Uses OpenAI’s Codex to provide code… Using AI to better code, debug and manage projects
I have experiment and I want to put the best result in bold in Latex, but it takes time to compare values and identify the smallest values in each row. So, I can ask ChatGPT… Analyze experiment results faster with ChatGPT
I advised a master’s student to use the binomial probability formula to determine the likelihood of attracting the affection of 15 girls, with Cupid’s success rate at 0.7. The analysis shows that the highest probability of success occurs when 10 girls reciprocate love, with a probability of 0.33.
Supplementary materials for section Grazing the maze of probability & A random variable mood in the KSML app: Basic rules of probability: Mutually exclusive events Conditional probability for medical testing in a forestThe conditional probability… Grazing the maze of probability
Type into ChatGPT: And you may see something like this So, you basically typed something really simple, and ask ChatGPT to do the annoying math formatting for you ?.
Connecting matrices to systems of linear equations can indeed help in better understanding and remembering the properties of matrices. By visualizing how matrix operations correspond to operations on systems of equations, abstract matrix properties become… Tricks for remembering elementary matrix operations
Graphical Lasso, also known as GLasso, is a statistical technique used for estimating the sparse inverse covariance matrix (precision matrix) of a multivariate Gaussian distribution. Here, Sparsity means that many elements of the matrix are… Estimating the sparse inverse covariance matrix (precision matrix) by Graphical Lasso (with Python implementation)
Generating missing values Generating missing values with a given percentage of missingness for a dataframe or numpy array: Generating missing values with a given missing rate for a time series list: Calculating MSE ignoring missing… Generating missing data and evaluating missing data analysis in Python