text:
# New_York_City_Green_Taxi

![x](Images/Boro-Taxi-4.png)

Contents of the repo

1 .[The report](https://github.com/kayambi/Taxi_Cab_of_New_York/blob/master/README.md) in .md format in the subdir/Taxi_Cab_of_New_York/Taxi_Cab_of_New_York.md

2. New York City Green Taxi notebook[Transportation project work in progress.ipynb](https://github.com/kayambi/Taxi_Cab_of_New_York/blob/master/DataSet/Transportation%20project%20work%20in%20%20progress%20.ipynb)

```Python
import pandas as pd
import csv
import numpy as np
from datetime import datetime
import datetime as dt
from datetime import timedelta  # it is a spanned based time .
from datetime import datetime, date, time
from datetime import date

import matplotlib.pyplot as plt
import os, json, requests, pickle
from scipy.stats import skew
from shapely.geometry import Point,Polygon,MultiPoint,MultiPolygon
from scipy.stats import ttest_ind, f_oneway, lognorm, levy, skew, chisquare

#import scipy.stats as st
from sklearn.preprocessing import normalize, scale
from tabulate import tabulate #pretty print of tables. source: http://txt.arboreus.com/2013/03/13/pretty-print-tables-in-python.html
from shapely.geometry import Point,Polygon,MultiPoint
%matplotlib inline
import warnings
warnings.filterwarnings('ignore')
! pip install shapely
! pip install shapely[vectorized]
!pip install git-lfs
from scipy.stats import chisquare

# Analysis

In this notebook, I will explore data on New York City Green Taxi of [september 2015](http://www.nyc.gov/html/tlc/html/about/trip_record_data.shtml). I will start with some warm up questions about the dataset. Later, I will build a model to predict the percentage tip a driver would exepect on each trip. The code is fully written in python with few additional open-source libraries easy to install.

- [shapely](https://pypi.python.org/pypi/Shapely)
- [scikit learn](http://scikit-learn.org/stable/)
- [tabulate](http://txt.arboreus.com/2013/03/13/pretty-print-tables-in-python.html)

In this analysis, some notion of statistics and hypothesis test are used but are very easy to follow. This [handbook of statistics](http://www.biostathandbook.com/index.html) can be used as a reference to explain basics.

# Start With Warm up

*** Firs Step . Download Dataset and Print out the size. ***

``` python

# Download the september 2019 dataset

if os.path.exists('Data_Sptember.csv'):# check if the Data is present and load it .
    data= pd.read_csv('Data_Sptember.csv')
# else: # Download Data Jan if not available on desktop .
#     url = "https://s3.amazonaws.com/nyc-tlc/trip+data/green_tripdata_2019-01.csv"
#     data= pd.read_url('url')
#     data= Data.to_csv(url.split('/')[-1]

# print number rows and number of columns
print("Number of rows:",data.shape[0])
print("Number of columns:",data.shape[1])

Number of rows: 1048575
Number of columns: 21

### Let`s take a loo at the distribution of the trip distance.

``` Python

# define the figure with 2 subplots
fig,ax = plt.subplots(1,2,figsize = (15,4))

# Histogram of Number of Trip distance

data.Fare_amount.hist(bins=30 , ax = ax[0])
ax[0].set_xlabel('fare_amount (per mile)')
ax[0].set_ylabel('Count')
ax[0].set_yscale('log')
ax[0].set_title('A. Histogram of fare_amount With outliers included')

# Create a vector to contain  Trip Distance

v = data.Trip_distance
# # Exclude any data point that is further than 4 standard deviations of median point
# and plot hist with 40 bin

v[~((v-v.median()).abs()>4*  v.std())].hist(bins=40, ax= ax[1])
ax[1].set_xlabel('fare_amount (per mile)')
ax[1].set_ylabel('Trip Count')
ax[1].set_title(' g . Histogram of fare_amount without outliers')

# Apply a lognormal fit . use the mean of Trip Distancce as the scale parameter
Scatter,loc,mean = lognorm.fit(data.Trip_distance.values,
                               scale = data.Trip_distance.mean(),
                                loc = 0 )
pdf_fitted = lognorm.pdf(np.arange(0,14,.1), Scatter , loc, mean)
ax[1].plot(np.arange(0,14,.1), 500000 * pdf_fitted, 'r')
ax[1].legend (['Data', 'lognorm fit '])

# # # create a vector to contain Trip Distance
v = data.Trip_distance
# # exclude any data point located further than 3 standard deviations of the median point and
# # plot the histogram with 30 bins

# apply a lognormal fit. Use the mean of trip distance as the scale parameter

scatter,loc,mean = lognorm.fit(data.Trip_distance.values,
                               scale=data.Trip_distance.mean(),
                               loc=0)
pdf_fitted = lognorm.pdf(np.arange(0,12,.1),scatter,loc,mean)
ax[1].plot(np.arange(0,12,.1),600000 * pdf_fitted,'r')
ax[1].legend(['Lognormal_Fit', 'Data'])

# # export the figure
# plt.savefig('Question2.jpeg',format='jpeg')

plt.show()

```
![png](Images/Histograms.png)

The Trip Distance is asymmetrically distributed.It is skewed to the right and it has a median smaller than it`s mean and both smaller than the standard deviation. The skewness is due to the fact that the variable has a lower boundary of 0.The distance can't be negative. https://www.itl.nist.gov/div898/handbook/eda/section3/eda3669.htm

To the left is plotted the distribution of the entire raw set of Trip distance. To the right, outliers have been removed before plotting. *Outliers are defined as any point located further than 3 standard deviations from the mean*

** The hypothesis:** The trips are not random. If there were random, we would have a (symmetric) Gaussian distribution. The non-zero autocorrelation may be related the fact that people taking ride are pushed by a common cause, for instance, people rushing to work.

*** We are going to examine if the time of the day has any impact on the trip distance ***

```python

#First, convert pickup and drop off datetime variable in their specific right format

data['lpep_pickup_dt'] = data.lpep_pickup_datetime.apply(lambda x:dt.datetime.strptime(x,"%m/%d/%y %H:%M"))
data['lpep_dropoff_dt'] = data.	Lpep_dropoff_datetime.apply(lambda x:dt.datetime.strptime(x,"%m/%d/%y %H:%M"))

# Second, create a variable for pickup hours
data['Pickup_hour'] = data.lpep_pickup_dt.apply(lambda x:x.hour)

# Mean and Median of trip distance by pickup hour
# I will generate the table but also generate a plot for a better visualization

fig,ax = plt.subplots(1,1,figsize=(9,5)) # prepare fig to plot mean and median values
# use a pivot table to aggregate Trip_distance by hour
table1 = data.pivot_table(index='Pickup_hour', values='Trip_distance',aggfunc=('mean','median')).reset_index()
# rename columns
table1.columns = ['Hour','Mean_distance','Median_distance']
table1[['Mean_distance','Median_distance']].plot(ax=ax)
plt.ylabel('Metric (miles)')
plt.xlabel('Hours after midnight')
plt.title('Distribution of trip distance by pickup hour')
#plt.xticks(np.arange(0,30,6)+0.35,range(0,30,6))
plt.xlim([0,23])
plt.savefig('Question3_1.jpeg',format='jpeg')
plt.show()
print ('-----Trip distance by hour of the day-----\n')
print (tabulate(table1.values.tolist(),["Hour","Mean distance","Median distance"]))
```

![png](Images/TripDistribution.png)

-----Trip distance by hour of the day-----

  Hour    Mean distance    Median distance
------  ---------------  -----------------
     0          3.1677               2.23
     1          3.0324               2.14
     2          3.0466               2.16
     3          3.19384              2.22
     4          3.5175               2.37
     5          4.12593              2.89
     6          4.07374              2.86
     7          3.33637              2.22
     8          3.10675              2.035
     9          3.04123              2
    10          2.95583              1.94
    11          2.92087              1.89
    12          2.89317              1.89
    13          2.87991              1.85
    14          2.85606              1.82
    15          2.87486              1.82
    16          2.78882              1.8
    17          2.70006              1.79
    18          2.68297              1.8
    19          2.75097              1.87
    20          2.79266              1.9
    21          3.03323              2.06
    22          3.24032              2.22
    23          3.26282              2.26

![png](Images/Trip_Distribution1.png)

-----Trip distance by hour of the day-----

  Hour    Mean distance    Median distance
------  ---------------  -----------------
     0          3.29074              2.3
     1          3.15925              2.22
     2          3.14788              2.22
     3          3.21088              2.275
     4          3.45483              2.39
     5          4.19538              2.91
     6          3.97609              2.8
     7          3.25145              2.1
     8          2.97537              1.9
     9          3.06321              2.02
    10          3.0371               2
    11          2.87525              1.87
    12          2.90718              1.9
    13          2.88218              1.86
    14          2.76009              1.8
    15          2.78375              1.8
    16          2.78264              1.8
    17          2.71911              1.77
    18          2.69802              1.8
    19          2.78718              1.88
    20          2.80607              1.91
    21          2.97203              2.03
    22          3.17881              2.2
    23          3.29715              2.3

-> We observe long range trips in the morning and evenings. Are these people commuting to work? If so how do they get back home. The evening peak are shorter than the morning peak. I would hypothesize that people are okay to take cabs in the morning to avoid being late to their early appointments while they would take public transportation in the evening. However, this might not apply to NYC

***Let's also compare trips that originate (or terminate) from (at) one of the NYC airports. We can look at how many they are, the average fair, ...***

Reading through the dictionary of variables, I found that the variable RateCodeID contains values indicating the final rate that was applied. Among those values, I realized that there is Newark and JFK which are the major airports in New York. In this part, I will use this knowledge and group data with RateCodeID 2 (JFK) and 3 (Newark). - An alternative (which I didn't due to time constraint) is to (1) get coordinates of airports from google map or http://transtats.bts.gov (2) get at least 4 points defining a rectangular buffer zone near the airport (3) build a polygon shape using shapely [https://pypi.python.org/pypi/Shapely] and (3) check if any pickup/drop-off location coordinates is within the polygon using shapely again. This method was first tried but was found to be time consuming -

``` python

# Selecting airport trips, finding the average of fare per trip, total amount charged.

airports_trips = data[(data.RateCodeID== 1) | (data.RateCodeID == 3)]
print("Number of Trips to/ from NYC airports :" ,airports_trips.shape[0])
print("Average of Fare (calculated by meter) of trips to / from NYC airports: $ ",airports_trips.Fare_amount.mean(),"per trip")
print("Average Total Amount charged before the trips to / from NYC airports: $ ",airports_trips.Total_amount.mean(),"per trip")
print("Maximum of Fare (calculated by meter) of trips to / from NYC airports: $ ",airports_trips.Fare_amount.max(),"per trip")

Number of Trips to/ from NYC airports : 1020731
Average of Fare (calculated by meter) of trips to / from NYC airports: $  12.391516530799978 per trip
Average Total Amount charged before the trips to / from NYC airports: $  14.887327758421408 per trip
Maximum of Fare (calculated by meter) of trips to / from NYC airports: $  580.5 per trip

Now that we have seen the number and mean fare of airport trips, let's take a closer look at how trips are distributed by trip distances and hour of the day

``` python

# Create a vector to contain Trip Distance .
v1 = airports_trips.Trip_distance
v2 = data.loc[~data.index.isin (v1.index),'Trip_distance'] # non- airport trips

# remove outliers :
# Exclude any data point located further than 2 standard deviations of the median point and histogram with 30 bins

v1 = v1[~((v1-v1.mean()).abs()>2*v1.std())]
v2 = v2[~((v2-v2.mean ()).abs()>2*v2.std())]

# Define Bins boundries.

bins = np.histogram(v1, normed = True)[1]
h1 = np.histogram(v1, bins = bins , normed = True)
h2 = np.histogram(v2 , bins = bins , normed =True)

# # plot distribution of trips distance within groups

fig,ax = plt.subplots(1,2,figsize = (15,5))
w =.4 *(bins[1] - bins[0])
ax[0].bar(bins[:-1], h1[0], alpha =1 , width = w, color = 'y')
ax[0].bar(bins[:-1] + w, h2[0], alpha =1 , width = w, color = 'orange')
ax[0].legend(['Airport Trip','Non-airport trips '] , loc = 'best', title = 'Group')
ax[0].set_xlabel('Trip Distance(miles)')
ax[0].set_ylabel('Group Normalized Trips Count')
ax[0].set_title('A. Trip Distance Distribution')

# ax[0].set_yscale('log')

# Plot hourly distribution

airports_trips.Pickup_hour.value_counts(normalize=True).sort_index().plot(ax=ax[1])
data.loc[~data.index.isin(v1.index),'Pickup_hour'].value_counts(normalize=True).sort_index().plot(ax=ax[1])
ax[1].set_xlabel('Hours after midnight')
ax[1].set_ylabel('Group normalized trips count')
ax[1].set_title('B. Hourly distribution of trips')
ax[1].legend(['Airport trips','Non-airport trips'],loc='best',title='group')
# plt.savefig('Question3_2.jpeg',format='jpeg')
plt.show()

```

![png](Images/Hour_Dist.png)

A . The trip distance distribution shows two peaks . Airport trips follow
the same trend as the rest of trips for short trips (trip distance less than or equal 2 miles).
However , there is also an increased number for long range trips (18 miles) which might
correspond to a great number people coming to airports from further residential areas .
A check on a google map shows that the distance between JFK and Manhattan is about 18 miles whereas Newark to Manhattan is 15 miles.

B.The hourly distribution shows that the number of trips at airports peaks around 3 Pm while it peaks 2 hours later .
on other hand , there is shortage in airports riders at 2 Am while the rest of NYC goes completely down 3 hours later at 5 AM .

## Predictive model

In this section , I will take you through my analysis towards building a model to predict the percentage trip.

*** 1 . Let's build a derived variable for tips for a percentage of the total fare.***

Before we proceed with this , some cleaning are necessary . Since the initial charge for NYC green is $2.5 ,
any transaction with a smal total amount is invalid , thus it to be dropped.

``` python

data = data[(data.Total_amount >= 2.5)]

# Cleaning

data["Tip_percentage"] = 100 * data.Total_amount / data.Total_amount
print("Summary: Tip_percentage \n", data.Tip_percentage.describe())

Summary: Tip_percentage
count    1.043476e+06
mean     1.000000e+02
std      1.277300e-15
min      1.000000e+02
25%      1.000000e+02
50%      1.000000e+02
75%      1.000000e+02
max      1.000000e+02
Name: Tip_percentage, dtype: float64

***2. Similarly to the comparison between trips to /from airports with the rest of trips , it is worthy to spend more time and check wether trips originating from Upper Manhattan have different percentage tip.***

To identify trips originating from upper Manhattan :
    -From google map , collect latitude and longitude data of atleast 12 points that approximately define
    the bounding box of upper Manhattan .
    -Create a polygon using shapely.geometry.Polygon :[https://pypi.org/project/Shapely/]
    - Check if the polygon contain a location defined by (latitude, longitude)

``` python

# import library

from shapely.geometry import Point,Polygon,MultiPoint
# data points that define the bounding box of the Upper Manhattan

U_manhattan = [(40.796937, -73.949503),(40.787945, -73.955822),(40.782772, -73.943575),
              (40.794715, -73.929801),(40.811261, -73.934153),(40.835371, -73.934515),
              (40.868910, -73.911145),(40.872719, -73.910765),(40.878252, -73.926350),
              (40.850557, -73.947262),(40.836225, -73.949899),(40.806050, -73.971255)]

poi = Polygon(umanhattan)
poi

# Create a function to check if a location is located inside Upper Manhattan
def is_within_bbox(loc,poi=poi):

    """
    This function returns 1 if a location loc(lat,lon) is located inside a polygon of interest poi
    loc: tuple, (latitude, longitude)
    poi: shapely.geometry.Polygon, polygon of interest
    """

    return 1*(Point(loc).within(poi))
tic = dt.datetime.now()
print(tic)

# Create a new variable to check if a trip originated in Upper Manhattan
data['U_manhattan'] = data[['Pickup_latitude','Pickup_longitude']].apply(lambda r:is_within_bbox((r[0],r[1])),axis=1)
print("Processing Time :", dt.datetime.now() - tic)

 ```

2019-12-31 13:10:37.111900
Processing Time : 0:00:49.209181

- Compare distributions of the two groups

``` python

# Create a vector to contain Tip

v1 = data[(data.U_manhattan== 0) & (data.Tip_amount>0)].Tip_amount
v2 = data[(data.U_manhattan==1) & (data.Tip_amount>0)].Tip_amount

# generate bins and histogram values
bins = np.histogram(v1,bins=10)[1]
h1 = np.histogram(v1,bins=bins)
h2 = np.histogram(v2,bins=bins)

# generate the plot
First suplot: visualize all data with outliers
fig,ax = plt.subplots(1,1,figsize=(10,5))
w = .4*(bins[1]-bins[0])
ax.bar(bins[:-1],h1[0],width=w,color='r')
ax.bar(bins[:-1]+w,h2[0],width=w,color='g')
ax.set_yscale('log')
ax.set_xlabel('Tip amount (%)')
ax.set_ylabel('Count')
ax.set_title('Tip')
ax.legend(['Non-Manhattan','Manhattah'],title='origin')
plt.show()
print ('t-test results:', ttest_ind(v1,v2,equal_var=False))

```

![png](Images/Origin_Tip.png)

t-test results: Ttest_indResult(statistic=47.12962151545506, pvalue=0.0)

Then two distribution look the same however the t-test results in zero p-value to imply tha the groups are different
at 95 % level of comfidence

###  The Model
###  Summary

The initial dataset contained 1048575 transactions with 21 time-series, categorical and numerical variables. In order to build the final model, four phases were followed (1) data cleaning, (2) feature engineering (3) exploratory data analysis and (4) model creation

The cleaning consisted in drop zero variance variables (Ehail_fee), replacing invalid with the most frequent values in each categorical variable whereas the median was used for continuous numerical variables. Invalid values could be missing values or values not allowed for specific variables as per the dictionary of variables. In this part, variables were also converted in their appropriate format such datetime.

The feature engineering part created 10 new variables derived from pickup and dropoff locations and timestamps, trip distance.

During the exploration, each variable was carefully analyzed and compared to other variables and eventually the target variable, Percentage tip. All numerical variables were found to follow lognormal or power law distributions althouth there was found no linear relationship between numerical and the target variable. An interesting insight was uncovered in the distribution of the percentage tip. It was found that only 40% of the transactions paid tip. And 99.99% of these payments were done by credit cards. This inspired me to build the predictive model in two stages (1) classification model to find out weither a transaction will pay tip and (2) regression model to find the percentage of the tip only if the transaction was classified as a tipper. Another insight was that the most frequent percentage is 18% which corresponds to the usual restaurant gratuity rate.

With lack of linear relationship between independent and depend variables, the predictive model was built on top of the random forest regression and gradient boosting classifier algorithms implemented in sklearn after routines to optimize best parameters. A usable script to make predictions as attached to this notebook and available in the same directory.

**Note:** The code to make predictions is provided in the same directory as tip_predictor.py and the instructions are in the recommendation part of this section.

Following,each part of the analysis fully explained with accompanying python code

### 1 Data Cleaning

This part concerns work Done to treat invalid data.
- Ehail_fee was removed since 99% of the data are missing
- Missing values in trip_type were replaced with the most common value that was 1
-Invalid data were found in :
    -RateCodeID: about 0.01 % of the values were 99 . These were replaced by the most common value 2
    - Extra : 0.08% of the transactions had negative Extra. These were replaced by 0 as the most frequent
   - Total_amount , Fare_amount , improvement_surcharge , Tip_amount : 0.16% of values were negative . The cases were considered as being machine errors during the data entry . They were replaced by their absolute values . Furthermore, as the minimum Total_amount that is chargeable for any service is $ 2.5,, every transaction falling below that amount was replaced by the median value of the Total_amount 11.76.

The code is proved below

```python

# Define a function to clean a loaded dataset

def clean_data(adata):
    """
    This function cleans the input dataframe adata:
    . drop Ehail_fee [99% transactions are NaNs]
    . compute  missing values in Trip_type
    . replace invalid data by most frequent value for RateCodeID and Extra
    . encode categorical to numeric
    . rename pickup and dropff time variables (for later use)

    input:
        adata: pandas.dataframe
    output:
        pandas.dataframe

    """
    ## make a copy of the input
    data = adata.copy()
    ## drop Ehail_fee: 99% of its values are NaNs
    if 'Ehail_fee' in data.columns:
        data.drop('Ehail_fee',axis=1,inplace=True)

    ##  replace missing values in Trip_type with the most frequent value which is going to be 1
    data['Trip_type '] = data['Trip_type '].replace(np.NaN,1)

    ## replace all values that are not allowed as per the variable dictionary with the most frequent allowable value
    # remove negative values from Total amound and Fare_amount

    print ("Negative values found and replaced by their abs")
    print ("Total_amount", 100*data[data.Total_amount<0].shape[0]/float(data.shape[0]),"%")
    print ("Fare_amount", 100*data[data.Fare_amount<0].shape[0]/float(data.shape[0]),"%")
    print ("Improvement_surcharge", 100*data[data.improvement_surcharge<0].shape[0]/float(data.shape[0]),"%")
    print ("Tip_amount", 100*data[data.Tip_amount<0].shape[0]/float(data.shape[0]),"%")
    print ("Tolls_amount", 100*data[data.Tolls_amount<0].shape[0]/float(data.shape[0]),"%")
    print ("MTA_tax", 100*data[data.MTA_tax<0].shape[0]/float(data.shape[0]),"%")

    data.Total_amount = data.Total_amount.abs()
    data.Fare_amount = data.Fare_amount.abs()
    data.improvement_surcharge = data.improvement_surcharge.abs()
    data.Tip_amount = data.Tip_amount.abs()
    data.Tolls_amount = data.Tolls_amount.abs()
    data.MTA_tax = data.MTA_tax.abs()

# RateCodeID
    indices_oi = data[~((data.RateCodeID>=1) & (data.RateCodeID<=6))].index
    data.loc[indices_oi, 'RateCodeID'] = 2
# 2 = Cash payment was identified as the common method
    print (round(100*len(indices_oi)/float(data.shape[0]),2),"% of values in RateCodeID were invalid.--> Replaced by the most frequent 2")

# Extra
    indices_oi = data[~((data.Extra==0) | (data.Extra==0.5) | (data.Extra==1))].index
    data.loc[indices_oi, 'Extra'] = 0 # 0 was identified as the most frequent value
    print (round(100*len(indices_oi)/float(data.shape[0]),2),"% of values in Extra were invalid.--> Replaced by the most frequent 0")

# Total_amount: the minimum charge is 2.5, so I will replace every thing less than 2.5 by the median 11.76 (pre-obtained in analysis)
    indices_oi = data[(data.Total_amount<2.5)].index
    data.loc[indices_oi,'Total_amount'] = 11.76
    print (round(100*len(indices_oi)/float(data.shape[0]),2),"% of values in total amount worth <$2.5.--> Replaced by the median 1.76")

#Encode categorical to numeric (I avoid to use dummy to keep dataset small)
    if data.Store_and_fwd_flag.dtype.name != 'int64':
        data['Store_and_fwd_flag'] = (data.Store_and_fwd_flag=='Y')*1

#Rename time stamp variables and convert them to the right format
    print ("renaming variables...")
    data.rename(columns={'lpep_pickup_datetime':'Pickup_dt','Lpep_dropoff_datetime':'Dropoff_dt'},inplace=True)

    Converting time stamp to the right format .
    print("converting timestamps variables to right format ...")

    data['Pickup_dt'] = data.Pickup_dt.apply(lambda x :dt.datetime.strptime(x, "%m/%d/%y  %H:%M"))
    data['Dropoff_dt'] = data.Dropoff_dt.apply(lambda x :dt.datetime.strptime(x, "%m/%d/%y  %H:%M"))

    print ("Done cleaning")

    return data
```python

#Run code to clean the data

data = clean_data(data)
```
Negative values found and replaced by their abs
Total_amount 0.0 %
Fare_amount 0.0 %
Improvement_surcharge 0.0 %
Tip_amount 0.0 %
Tolls_amount 0.0 %
MTA_tax 0.0 %
0.0 % of values in RateCodeID were invalid.--> Replaced by the most frequent 2
0.0 % of values in Extra were invalid.--> Replaced by the most frequent 0
0.0 % of values in total amount worth <$2.5.--> Replaced by the median 1.76
renaming variables...
converting timestamps variables to right format ...
Done cleaning

### 2 Feature Engineering

In this step, I intuitively created new variables derived from current variables.

-Time variables: Week, Month_day(Day of month), Week_day (Day of week), Hour (hour of day), Shift_type (shift period of the day) and Trip_duration.The were created under the hypothesis that people may be willing to tip depending on the week days or time of the day. For instance, people are more friendly and less stressful to easily tip over the weekend. They were derived from pickup time
-Trip directions: Direction_NS (is the cab moving North to South?) and Direction_EW (is the cab moving East to West). These are components of the two main directions, horizontal and vertical. The hypothesis is that the traffic may be different in different directions and it may affect the riders enthusiasm to tipping. They were derived from pickup and drop-off coordinates
-Speed: this the ratio of Trip_distance to Trip_duration. At this level, all entries with speeds higher than 240 mph were dropped since this is the typical highest speed for cars commonly used as taxi in addition to the fact that the speed limit in NYC is 50 mph. An alternative filter threshold would be the highest posted speed limit in NYC but it might be sometimes violated.
-With_tip: This is to identify transactions with tips or not. This variable was created after discovering that 60% of transactions have 0 tip.
-As seen that using the mean of trips from Manhattan is different from the mean from other boroughs, this variable can be considered as well in the model building. A further and deep analysis, would be to create a variable of the origin and destination of each trip. This was tried but it was computationally excessive to my system. Here, coming from Manhattan or not, is the only variable to be used.

```python

# Function to run the feature engineering
def engineer_features(data):
    """
    This function create new variables based on present variables in the dataset adata. It creates:
    . Week: int {1,2,3,4,5}, Week a transaction was done
    . Week_day: int [0-6], day of the week a transaction was done
    . Month_day: int [0-30], day of the month a transaction was done
    . Hour: int [0-23], hour the day a transaction was done
    . Shift type: int {1=(7am to 3pm), 2=(3pm to 11pm) and 3=(11pm to 7am)}, shift of the day
    . Speed_mph: float, speed of the trip
    . Tip_percentage: float, target variable
    . With_tip: int {0,1}, 1 = transaction with tip, 0 transction without tip

    input:
        adata: pandas.dataframe
    output:
        pandas.dataframe
    """

    # make copy of the original dataset
    data = data.copy()

    # derive time variables
    print("deriving time variables...")
     reference week "First week of Septemeber in 2015"
     date(2003, 12, 29).isocalendar()

    ref_week = dt.datetime(2015,9,1).isocalendar()[1]

    data['Week'] = data.Pickup_dt.apply(lambda x:x.isocalendar()[1]- ref_week + 1 )
    data['Week_day'] = data.Pickup_dt.apply(lambda x:isocalendar()[2])
    data['Month_day'] = data.Pickup_dt.apply(lambda x:x.day)
    data.rename(columns ={'Pick_hour':'Hour'} , inplace =True)
    data['Hour'] = data.Pickup_dt.apply(lambda x:x.hour)

   # Create shift variable: 1= (7am to 3 pm) , 2 =(3pm to 11 pm) , 3 = (11 pm to 7 am )

    data['Shift_type']=np.NAN
    data.loc[data[(data.Hour >=7) & (data.Hour<15)].index,'Shift_type'] = 1
    data.loc[data[(data.Hour >=15) & (data.Hour<23)].index,'Shift_type'] = 2
    data.loc[data[data.Shift_type.isnull()].index,'Shift_type'] = 3

   # Trip Duration

    print("Deriving Trip_duration ....")
    data['Trip_duration'] = ((data.Dropoff_dt - data.Pickup_dt).apply(lambda x:x.total_seconds()/60.))

    print("Deriving direction variables...")
    # Create direction variables Direction_North_South (NS)
    #This is 2 if Taxi moving frm North to South , 1 in the opposite direction and = 0 otherwise.
    data['Direction_NS'] = (data.Pickup_latitude > data.Dropoff_latitude)* 1+1
    indices = data[(data.Pickup_latitude == data.Dropoff_latitude) & (data.Pickup_latitude !=0)].index
    data.loc[indeces, 'Direction_NS'] = 0
    Create direction variables of Direction_EW(East to West)
    data["Direction_EW"] = (data.Pickup_longitude > data.Dropoff_longitude) * 1+1
    indeces = data[(data.Pickup_longitude ==data.Dropoff_longitude) & (data.Pickup_longitude !=0)].index
    data.loc[indeces,'Direction_EW'] = 0
    # Create a variable for speed
    print("deriving Speed. make sure to check for possible NANS and inf vals...")
    data['Speed_mph'] = data.Trip_distance / (data.Trip_duration / 60)
    # replace all NANs values and values > 240mph by values samples from a random distribution of mean 12.9  and standard deviation          6.8 mph. These Values were extracted from the distribution
    indeces_oi = data[(data.Speed_mph.isnull()) / (data.Speed_mph>240)].index
    data.loc[indeces_oi, 'Speed_mph'] = np.abs(np.random.normal(loc=12.9, scale =6.8, size= len(indices_oi)))
    print("Greate we are done with Feature Engineering ! :-")
    Create a new a variable to check if the trip originated in Upper Manhattan
    print("Checking where the trip originated")
    data[U_manhattan] = data[['Pickup_latitude','Pickup_longitude']]. apply(lambda r: is_within_bbox((r[0], r[1])), axis=1)
    # Create a tip percentage variable
    data['Tip_percentage'] = 100*data.Tip_amount / data.Total_amount
    # Create with tip variable
    data['With_tip'] =(data.Tip_percentage > 0)*1

    return data

# collected bounding box points

umanhattan = [(40.796937, -73.949503),(40.787945, -73.955822),(40.782772, -73.943575),
              (40.794715, -73.929801),(40.811261, -73.934153),(40.835371, -73.934515),
              (40.868910, -73.911145),(40.872719, -73.910765),(40.878252, -73.926350),
              (40.850557, -73.947262),(40.836225, -73.949899),(40.806050, -73.971255)]

poi = Polygon(umanhattan)
# create a function to check if a location is located inside Upper Manhattan
def is_within_bbox(loc,poi=poi):
    """
    This function checks if a location loc with lat and lon is located within the polygon of interest
    input:
    loc: tuple, (latitude, longitude)
    poi: shapely.geometry.Polygon, polygon of interest
    """
    return 1*(Point(loc).within(poi))

```

```python

# run the code to create new features on the dataset

print "size before feature engineering:", data.shape
data = engineer_features(data)
print "size after feature engineering:", data.shape
```

    size before feature engineering: (1494926, 20)
    deriving time variables...
    deriving Trip_duration...
    deriving direction variables...
    deriving Speed. Make sure to check for possible NaNs and Inf vals...
    Feature engineering done! :-)
    checking where the trip originated...
    size after feature engineering: (1494926, 32)

``` python

# Uncomment to check for data validity.

data.describe().T

![png](Images/Tdescribe.png)

```

### 3 .Exploratory Data Analysis

This was the key phase of the analysis . A look at the distribution of the target
variable , Tip_percentage showed that 60 % of all transaction did not give tip(See Figure below , left).
A second tip at 18 % corresponds to the usual NYC customary
gratuity rate which fluctuates between 18 % and 25 % (See Figure below , right).
Based on the information , the model can be built in two steps .

1.First Step: Create classification model to predict wether tip will be given or not . Here a new variable " with_tip" of 1 (if there is tip ) and 0 (otherwise) was created.
* Second Create regression model for transaction with non-zero tip.

2.The two distributions look the same however the t-test results in a zero p-value to imply that the two groups are  different at  92% level of confidence.

```python

##Code to compare the two tips_percentage indentifed groups
code to compare the two Tip_percentage identified groups
# split data in the two groups

data1= data [data.Tip_percentage > 0]
data2 = data[data.Tip_percentage == 0 ]

# generate histograms to compare

fig,ax = plt.subplots(1,2,figsize =(14,4))
data.Tip_percentage.hist(bins =10,normed=True, ax=ax[0] , color ='r')
ax[0].set_xlabel('Tip(%)')
ax[0].set_title('Distribution of Fare_amount (%) - All transaction')
data1.Tip_percentage.hist(bins =300,normed=True,ax=ax[1], color ='y')
ax[1].set_xlabel('Tip (%)')
ax[1].set_title('Distribution of Tip (%) - Transaction with tips')
ax[1].set_ylabel('Group normed count')

plt.savefig('Question4_target_varc.jpeg',format='jpeg')

plt.show()
```

![png](Images/DistribFAmount.png)

Next, each variable distribution and its relationship with the Tip percentage were explored. Few functions were implemented to quickly explore those variables:

```python

# Functions For Exploratory Data Analysis
def visualize_contious(df, label, method={'type':'histogram','bins':20},outlier ='on'):

    The function to quickly visualize continuos variables
    df:pandas.dataFrame
    label:str, name of variable to be plotted . it should be present in df.columns
    method:dict, contains info of the type of plot to generate . It can be histogram or boxplot
    [-Not yet developped]
    outlier:{'on','off'}, set it off you need to cut off outliers .
    Outliers are all those points
    Located at 3 standard deviations further from the mean.

    # Create Vector of variable of interest
    v = df[label]
    #Define mean and standard deviation
    m = v.mean()
    s = v.std()
    #Prepare the figure

    fig, ax = plt.subplots (1, 2, figsize=(14,4))
    ax[0].set_title('Distribution of' + label)
    ax[1].set_title('Tip % by '+ label)
    if outlier == 'off':   # remove outliers accondingly and update titles
        v = v[(v-m) < 3 * s]
        ax[0].set_title('Distribution of' + label + '(no outliers)')
        ax[1].set_title('Tip % by' + label+'(no outliers)')
    if method['type'] == 'histogram': # plot the histogram
        v.hist(bins =method['bins'], ax=ax[0])
    if method['type'] == 'boxplot' : # plot the box plot
        df.loc[v.index].boxplot(label, ax =ax[0])
    ax[1].plot(v,df.loc[v.index].Tip_percentage,'.',alpha = 0.4)
    ax[0].set_xlabel(label)
    ax[1].set_xlabel(label)
    ax[0].set_ylabel('Count')
    ax[1].set_ylabel('Tip (%)')

def visualize_categories(df, CatName ,chart_type = 'Histogram',ylimit =[None,None]):

    """
    This Functions Helps to quickly visualize categorical variables .
    this functions calls other functions generate_boxplot and generate_histogram
    df:pandas.DataFrame
    chart_type : {histogram, boxplot}, choose which type of chart to plot
    ylim:tuple, list.Valid if chart_type is histogram
    """

    print(catName)
    cats= sorted(pd.unique(df[catName]))
    if chart_type == 'boxplot': # generate boxplot
        generate_boxplot(df, catName , ylimit)
    elif chart_type == 'histogram': # generate histogram
        generate_histogram(df, catName)
    else:
        pass
    # calculate test Stastics
    groups = df[[catName ,'Tip_percentage']].groupby(catName).groups # create groups
    tips = df.Tip_percentage
    if len(cats) <= 'catName': # if there are only two groups use t-test
        print(ttest_ind(tips[groups[cats[0]]], tips[groups[cats[1]]]))
    else: # otherwise the command to be evaluated
        cmd = "f_oneway("
        for cat in cats :
            cmd +="tips[groups["+str(cat)+"]],"
            cmd = cmd[:-1]+")"
            print("one way anova test:" , eval(cmd)) # Evaluate the command and print
        print("frequency of categories (%):\n", df[catName].value_counts(normalize = True)*100)

def test_classification(df, label, y1=[0,50]):
    """
    This Function test if the means of the two groups with_tip and without tip are different
    at 95 % of confidence level.
    It will also generate a box plot of the viariable by tipping groups
    label:str ,label to test
    y1:tuple or list (default = [0,50]) , y limits on the ylabel of the boxplot
    df:pandas.DataFrame(deafult =data)

    Example:run <visualize_continous(data,'Fare_amount', outlier ='on ') >

    """

    if len(pd.unique(df[label])) == 2: # check if the variable is categorical with only two
 categories  and run the test .
        vals = pd.unique(df[label])
        gp1 =df[df.With_tip == 0][label].value_counts().sort_index()
        gp2 =df[df.With_tip == 1][label].value_counts().sort_index()
        print("t-test if ", label , "can be used to distinguish transanctionwith tip and without tip")
        print("chisquare(gp1,gp2)")
    elif len(pd.unique(df[label])) >= 10 : # otherwise run the t-test
        df.boxplot(label, by='With_tip')
        plt.ylim(y)
        plt.show
         print("t-test if, label , can be used to distinguish transaction with tip and without  " )
        print("results:", ttest_ind(df[df.WIth_tip == 0][label].value, df[df.With_tip ==1][label].values , False))
    else:
         passs

def generate_boxplot(df,catName,ylimit):
    """
    generate boxplot of tip percentage by variable "catName" with ylim set ylimit
    df:pandas.Dataframe
    catName :str
    ylimit :tuple , list
    """
    df.boxplot('Tip_percentage', by=CatName)
    # plt.title('Tip % by ' +catName )
    plt.title('')
    plt.ylabel('Tip (%)')
    if ylimit != [None, None]:
        plt.ylim(ylimit)
    plt.show()

def generate_histogram(df, catName):

    """
    generate histogram of tip percentage by varible "CatName" with ylimit set ylimit
    df: pandas.dataFrame
    catName : str
    ylimit : tuple , list

    """

    cats = sorted(pd.unique(df[catName]))
    colors =plt,cm.jet(np.linspace(0,1,len(cats)))
    hx = np.array(map(lambda x : round(x, 1 ), np.histogram(df.Tip_percentage,bins=20)[1]))
    fig , ax = plt.subplots(1,1,figsize=(15, 4))
    for i , cat in enumerate (cats):
        vals =df [df[catName] == cat ].Tip_percentage
        h=np.histogram(vals, bins =hx)
        w =0.9*(hx [:-1]+ w*i, h[0], colors == colors[i], width == w)
    plt.legend(cats)
    plt.ysccale('log')
    plt.title ('Distribution of Tip by' + catName )
    plt.xlabel ('Tip (%)')

    plt.show()
```
Starting with continuous variables, two main insights were discovered: A lognormal-like or power law distribution of the Fare amount and a non-linear function of the tip percentage as function of the total amount. The tip percentage decreases as the fare amount increases but converges around 20%. The density of scattered points implies that there is a high frequency of smaller tipps at low Fare_amount. Can we say that people restrain themselves to tipping more money as the cost of the ride becomes more and more expensive? Or since the fare grows with the length of the trip and trip duration, can we say that riders get bored and don't appreciate the service they are getting? Many questsions can be explored at this point

```python

#Example of exploration of the Fare_amount using the implemented:
visualize_contious(data1 , 'Fare_amount', outlier ='on')

```

![png](Images/duration_cluster.png)


text:
# Running the compressor tools from python

Both the automatic and manual tools use PyO3 to allow the compressor
to be run from Python.

To see any output from the tools, logging must be setup in Python before
the compressor is run.

## Setting things up

1. Create a virtual environment in the place you want to use the compressor from
(if it doesn't already exist)
`$ virtualenv -p python3 venv`

2. Activate the virtual environment and install `maturin` (if you haven't already)
`$ source venv/bin/activate`
`$ pip install maturin`

3. Navigate to the correct location
For the automatic tool:
`$ cd /home/synapse/rust-synapse-compress-state/synpase_auto_compressor`
For the manual tool:
`$ cd /home/synapse/rust-synapse-compress-state`

3. Build and install the library
`$ maturin develop`

This will install the relevant compressor tool into the activated virtual environment.

## Automatic tool example:

```python
import synapse_auto_compressor

synapse_auto_compressor.compress_state_events_table(
  db_url="postgresql://localhost/synapse",
  chunk_size=500,
  default_levels="100,50,25",
  number_of_chunks=100
)
```

# Manual tool example:

```python
import synapse_compress_state

synapse_compress_state.run_compression(
  db_url="postgresql://localhost/synapse",
  room_id="!some_room:example.com",
  output_file="out.sql",
  transactions=True
)
```

repository: matrix-org/rust-synapse-compress-state

branch_name: refs/heads/main

extension: md

gha_language: Rust

license_type: permissive

repo_name: matrix-org/rust-synapse-compress-state


text:
# mc-exercise
# Problem Domain
Merge conflicts occur when files get out of sync between GitHub and a user’s local code base, and Git becomes unsure of where HEAD really is.

Let’s describe and experience how a pretend team would get into that situation and then extricate themselves from it.

Results from exercise:
1. It is always important to make sure that you are working in your own local branch instead of master to avoid conflicts
2. Always do a git pull when starting to make sure that you are working on up-to-date content.
3. Whenever someone does a push everyone should stop and do a pull to help avoid merge conflicts.

license: MIT

branch_name: refs/heads/master

gha_license_id: MIT


text:
![logo](/images/logo.png)

[![Build Status](https://travis-ci.org/jamesbowman/i2cdriver.svg?branch=master)](https://travis-ci.org/jamesbowman/i2cdriver)

I2CDriver is a tool for controlling any I2C device from your PC's USB port,
and can also monitor and capture I2C traffic.
It connects as a standard USB serial device, so there are no drivers to install.
The serial protocol is [very simple](/protocol.md),
and there are included drivers for

* Windows/Mac/Linux GUI
* Windows/Mac/Linux command-line
* Python 2 and 3
* Windows/Mac/Linux C/C++

![front](/images/hero.jpg)

Full documentation is at
[i2cdriver.com](http://i2cdriver.com).


text:
# Swoole Bundle Configuration

Documentation of available configuration parameters. See also symfony [bundle configuration](./../src/Bridge/Symfony/Bundle/DependencyInjection/Configuration.php) file or [swoole documentation](https://github.com/swoole/swoole-docs/tree/master/modules).

- [Swoole Bundle Configuration](#swoole-bundle-configuration)
  - [HTTP Server](#http-server)

## HTTP Server

*Example*:

```yaml
swoole:
  http_server:
    port: 9501
    host: <IP_ADDRESS>
    running_mode: process
    socket_type: tcp
    ssl_enabled: false
    trusted_hosts: localhost,<IP_ADDRESS>
    trusted_proxies:
      - '*'
      - <IP_ADDRESS>/8
      - 192.168.2./16

    # enables static file serving
    static: advanced
    # equals to:
    # ---
    # static:
    #     public_dir: '%kernel.project_dir%/public'
    #     strategy: advanced
    #     mime_types: ~
    # ---
    # strategy can be one of: (default) auto, off, advanced, default
    #   - off: turn off feature
    #   - auto: use 'advanced' when debug enabled or not production environment
    #   - advanced: use request handler class \K911\Swoole\Server\RequestHandler\AdvancedStaticFilesServer
    #   - default: use default swoole static serving (faster than advanced, but supports less content types)
    # ---
    # mime types registration by file extension for static files serving in format: 'file extension': 'mime type'
    # this only works when 'static' strategy is set to 'advanced'
    #
    #   mime_types:
    #       '*': 'text/plain' # fallback override
    #       customFileExtension: 'custom-mime/type-name'
    #       sqlite: 'application/x-sqlite3'

    # enables hot module reload using inotify
    hmr:
      enabled: auto
    # hmr enabled can be one of: off, (default) auto, inotify, external
    #   - off: turn off feature
    #   - auto: use inotify if installed in the system
    #   - inotify: use inotify
    #   - external: dump files included before server start to text files,
    # files are parsed and used in swoole entrypoint command to decide if hard/soft reload is needed
    # files location, usually %kernel.cache_dir%/swoole_bundle/
    #
    #    hmr:
    #        enabled: external
    #        file_path: '%swoole_bundle.cache_dir%'

    # enables api server on specific port
    # by default it is disabled (can be also enabled using --api flag via cli)
    api: true
    # equals to:
    # ---
    # api:
    #     enabled: true
    #     host: <IP_ADDRESS>
    #     port: 9200

    # additional swoole symfony bundle services
    services:

      # see: \K911\Swoole\Bridge\Symfony\HttpFoundation\TrustAllProxiesRequestHandler
      trust_all_proxies_handler: true

      # see: \K911\Swoole\Bridge\Symfony\HttpFoundation\CloudFrontRequestFactory
      cloudfront_proto_header_handler: true

      # see: \K911\Swoole\Bridge\Upscale\Blackfire\WithProfiler
      blackfire_profiler: false

      # see: \K911\Swoole\Bridge\Upscale\Blackfire\WithApm
      blackfire_monitoring: false

      # see: \K911\Swoole\Bridge\Tideways\Apm\WithApm
      tideways_apm:
        enabled: true
        service_name: 'app_name' # service name for Tideways APM UI

      # enable swoole logging
      access_log:
          enabled: false
          # define custom format or use some from AccessLogFormatter
          format: !php/const K911\Swoole\Bridge\Log\AccessLogFormatter::FORMAT_COMMON_TIME
          # whatever register monolog line formatter service
          register_monolog_formatter_service: true
          # service id, can be used in monolog -> handlers -> formatter
          monolog_formatter_service_name: 'monolog.formatter.line.swoole.access_log'
          # custom line formatter format, extra section is used by monolog processors
          monolog_formatter_format: "%%message%% %%context%% %%extra%%\n"

    # swoole http server settings
    # see https://openswoole.com/docs/modules/swoole-server/configuration
    settings:
      reactor_count: 2
      worker_count: 4
      # when not set, swoole sets these are automatically set based on count of host CPU cores

      log_level: auto
      # can be one of: (default) auto, debug, trace, info, notice, warning, error
      #   - auto: when debug set to debug, when not set to notice
      #   - {debug,trace,info,notice,warning,error}: see swoole configuration

      log_file: '%kernel.logs_dir%/swoole_%kernel.environment%.log'
      pid_file: /var/run/swoole_http_server.pid

      buffer_output_size: 2097152
      # in bytes, 2097152b = 2 MiB

      package_max_length: 8388608
      # in bytes, 8388608b = 8 MiB

      worker_max_request: 0
      # integer >= 0, indicates the number of requests after which a worker reloads automatically
      # This can be useful to limit memory leaks
      worker_max_request_grace: ~
      # 'grace period' for worker reloading. If not set, default is worker_max_request / 2. Worker reloads
      # after 'worker_max_request + rand(0,worker_max_request_grace)' requests

  task_worker: # task workers' specific settings
      services:
        reset_handler: true # default true, set to false to disable services resetter on task processing end
      settings:
        worker_count: 2 # one of: positive number, "auto", or null to disable creation of task worker processes (default: null)
  platform:
    coroutines:
      enabled: false
      # default false. when enabled, swoole coroutine hooks for IO apis get activated
      # (https://openswoole.com/docs/modules/swoole-coroutine-enableCoroutine) and all stateful services
      # are being used in contextual way with multiple instances for each service
      # (https://openswoole.com/article/isolating-variables-with-coroutine-context)
      max_coroutines: 100
      # default value is 100000, if not set
      # (https://openswoole.com/docs/modules/swoole-server/configuration#max_coroutine)
      max_concurrency: 100
      # default value is null, if not set
      # it helps limitting this number, so you can be sure, that there is a limited amount of proxified service instances
      # (https://openswoole.com/article/v4-7-1-released)
      max_service_instances: 100
      # default is the same as max_concurrency, if not set, or max_coroutines if max_concurrency is not set
      # it is important to limit the amount of stateful service instances, that need to be created,
      # otherwise there would be at least so many service instances as coroutines, which might be quite enough
      # regarding how much coroutines can be run concurrently (e.g. 100000)
      # this number is by default equal to max_concurrency or max_coroutines, since it doesn't make sense to have
      # more stateful services than coroutines, but sometimes it might have sense to have less instances (e.g. if max_coroutines is too high)
      stateful_services:
        - SomeStatefulServiceId
          # add stateful service ids which need to be proxified and the bundle cannot detect them alone
          # check the section below about coroutines usage
      compile_processors:
        - class: ProcessorClass1
          priority: 10
          config: [] # all data will be propagated to the processor constructor, this attribute is not needed
        - ProcessorClass2 # default priority
          # register classes implementing the CompileProcessor interface
          # check the section below about coroutines usage
      # configuration options for doctrine processor - set instance limits for each connection type, or global limit
      doctrine_processor_config:
        # max connections in each swoole process for each doctrine connections, default is max_service_instances
        global_limit: 10
        limits:
          # connection with name 'default' will have max 9 instances per swoole process, if not set, default is global_limit
          default: 9
```

## Additional info for coroutines usage

**WARNING!!! Coroutines usage in Symfony is highly EXPERIMENTAL at this stage. It still needs to be tested properly.
There may be changes in the configuration/implementation in later releases.**

Current implementation wupports coroutines in the web server and in task workers,
but no one has ever tested this approach in real life application. The feature was merged because it
was easier for maintenance. It is not recommended to use this feature in production yet.

To be able to use coroutines in a Symfony app, it is mandatory to change all stateful services of the app into special
proxies, which preserve separate state for each coroutine context. This bundle has a Proxifier service, which modifies
the Symfony container for each stateful service. The proxification phase happens during application compiling
(in a compiler pass).

There are some generally known services registered, which need to be proxified. Additionally, all services implementing
the `Symfony\Contracts\Service\ResetInterface` are proxified as well. For convenience it is also possible to register
stateful services to be processed by the bundle using 3 ways:
 - using `coroutines_support.stateful_services` configuration option
 - using compile processors
 - using the `swoole_bundle.stateful_service` service tag

**ANOTHER WARNING!!! This feature is dependent on the PHP FFI extension and a custom fork of `lisachenko/z-engine`,
The original package seems to [not be maintained anymore](https://github.com/lisachenko/z-engine/issues/55).
Feel free to reconsider usage of coroutines.**

The Z-Engine library is used to remove the `final` flag from classes in PHP runtime, because to make a proxy
for a class, tha class cannot be final by design. This is a working solution for final classes from 3rd party
libraries. Other solutions are considered, but none of them seems to be better than this one. Any better ideas
are appreciated.

To work as intended, please use `pixelfederation/z-engine`.

### Application kernel modification

To be able to use coroutines in your application the following trait has to be used in the application kernel class:
`K911\Swoole\Bridge\Symfony\Kernel\CoroutinesSupportingKernelTrait`.

This trait will disable state resetting of the app kernel while cloning it and makes some default overrides
and initializations.

### Proxification

To be able to run a Symfony app with Swoole coroutines without modifications of the app, this bundle implements
a technique, that identifies services critical for concurrent access (typically resettable services with request
scoped data) and enhances them to be safe for concurrent workload. It is done wia proxification this bundle
wraps the services to special proxies using the DI container. These proxies behave exactly as the same instance
types as the wrapped services, but under the hood, the proxies have a separate service instance for each coroutine.
This helps to hide and separate contextually scoped coroutine data under one service instance. With this technique,
it is possible to achieve a correct non-blocking workloads with coroutines within Symfony apps without
any code modification.

### Compile processors

Compile processors are project/other-bundle level extensions where it is possible to setup proxification
of custom services unknown to the proxification mechanism in this bundle. The intention is to have a programmatic
way to proxify chosen services or to add special tags to services ina a dynamic way.

All compile processors have to implement the interface
`K911\Swoole\Bridge\Symfony\Bundle\DependencyInjection\CompilerPass\StatefulServices\CompileProcessor`.
Compile processors run in the proxification phase of the `StatefulServicePass` compiler pass at the end of application
compilation.

Two services are accessible in the compile processor:
- Symfony container builder (you can use it the same way as in casual compiler passes)
- proxifier from this bundle (you can use it to proxify any service directly)

This is an example compile processor:

```php
<?php

declare(strict_types=1);

namespace K911\Swoole\Tests\Fixtures\Symfony\TestBundle\DependencyInjection\CompilerPass;

use K911\Swoole\Bridge\Symfony\Bundle\DependencyInjection\CompilerPass\StatefulServices\CompileProcessor;
use K911\Swoole\Bridge\Symfony\Bundle\DependencyInjection\CompilerPass\StatefulServices\Proxifier;
use K911\Swoole\Tests\Fixtures\Symfony\TestBundle\Service\SleepingCounter;
use Symfony\Component\DependencyInjection\ContainerBuilder;

final class SleepingCounterCompileProcessor implements CompileProcessor
{
    public function process(ContainerBuilder $container, Proxifier $proxifier): void
    {
        // this line can be achieved by registering FQCN to the coroutines_support.stateful_services configuration
        $proxifier->proxifyService(SleepingCounter::class);
    }
}
```

### Special tags

This bundle provides 5 special tags for marking some services to be handled specially, so they can safely run
in multiple coroutine contexts:

- `swoole_bundle.stateful_service` - use to mark a service to be proxified
- `swoole_bundle.decorated_stateful_service` - use to mark a decorated service to be proxified. (the bundle needs to guess
  the decorated service in the decorated chain, which needs to be proxified, and there may be changes in this feature).
- `swoole_bundle.safe_stateful_service` - use to mark a resettable service as safe for coroutine usage, so it doesn't need to be proxified
- `swoole_bundle.unmanaged_factory` - use to mark services which create other stateful services (-> factories) outside of the SF container context,
  e.g. in runtime (the point is to proxify those services in runtime)
- `swoole_bundle.stability_checker` - use this tag to mark a service as a stability checker

### Resetters

This bundle overrides Symfony resetting mechanism (`kernel.reset` tag), because the original mechanism is not well suited
for concurrent request processing. The main problem is that Symfony service resetter resets each already initialised
service on each request, which would lock a service instance of each service for each coroutine serving a request.
But not all of the services may be needed to be available for the request. Such behaviour might produce unnecessary
slowdowns while handling requests, since there are srvice instance limits implemented into the bundle. That means
that requests that don't need specific services would need to wait for other requests (which may or may not need
the same service instances as the waiting/blocked request), until they release the specific service instance.

This behaviour is problematic, so this bundle activates the resetting mechanism on each service instance only when
it is used for the first time in the request (while assigning it for concrete coroutine in the service pool). When
the request does not need the specific service, it won't get assigned to the coroutine. This helps the app to serve requests
without unnecessary blocking, e.g. when there are liveliness probes in the app that do not touch any database,
there is no need to assign a database connection for them. Since database connections (and maybe some other services)
are expensive resources, they will be used on demand, not always.

By default, this bundle automatically extracts all the resetter methods from Symfony service resetter and assigns
them to the service pools for each resettable service. That means, there is no need to do anything special in the app.

Sometimes there are special cases that emerge for using Symfony with coroutines turned on, like pinging DBAL connections
before the first query on each request (because the connections may be already closed, btw this bundle already has
a solution for this, using connection pingers
from [pixelfederation/doctrine-resettable-em-bundle](https://github.com/pixelfederation/doctrine-resettable-em-bundle)).

For special cases like this, you can implement a custom service resetter. The resetter has to implement
the `K911\Swoole\Bridge\Symfony\Container\Resetter` interface and has to be registered in the SF container
as a service. After that, you can configure any stateful service to use the resetter just by adding the resetter
service id to the stateful service tag like this:

```yaml
services:
  my_custom_resetter:
    class: My\Custom\ResetterClass

  some_stateful_service:
    class: My\Stateful\ServiceClass
    tags: [{ name: swoole_bundle.stateful_service, resetter: my_custom_resetter, reset_on_each_request: true}]

  some_SAFE_stateful_service:
      class: My\Stateful\SafeServiceClass
      tags: [{ name: swoole_bundle.safe_stateful_service, reset_on_each_request: true}]

  some_unmanaged_factory:
    class: My\Unmanaged\FactoryClass2
    tags: [{ name: swoole_bundle.unmanaged_factory, resetter: my_custom_resetter}]
```

By default, each resetted service is being reset on first usage during request. But there are some services,
that need to be reset on each request. To activate this, use the `reset_on_each_request` attribute.

### Stability checkers

Stability checkers are services services which make run-time checks for paired stateful services. They check,
if the paired service is stable and is able to be used for the next request (e.g. it can check if the entity manager is still open).
A stability checker usually doesn't have to be tagged manually, autoconfiguration will tag all services implementing
the `K911\Swoole\Bridge\Symfony\Container\StabilityChecker` interface automatically.

An example of a stability checker:

```php
<?php

declare(strict_types=1);

namespace K911\Swoole\Bridge\Doctrine\ORM;

use Doctrine\ORM\EntityManager;
use K911\Swoole\Bridge\Symfony\Container\StabilityChecker;
use UnexpectedValueException;

final class EntityManagerStabilityChecker implements StabilityChecker
{
    public function isStable(object $service): bool
    {
        if (!$service instanceof EntityManager) {
            throw new UnexpectedValueException(\sprintf('Invalid service - expected %s, got %s', EntityManager::class, \get_class($service)));
        }

        return $service->isOpen();
    }

    public static function getSupportedClass(): string
    {
        return EntityManager::class;
    }
}
```

When the checker returns false, the coroutine engine will forget the unstable service and will fetch a fresh instance
from the DI container. Otherwise the engine will mark the service instance as unused and will assign it to a coroutine
in the next web request.

## Opcache problems

This bundle overrides generated Symfony container class, it renames the original container class (and corresponding filename)
and generates a new container class which extends the original class.

Including these two relatively big classes into the engine at runtime somehow corrupts the Opcache
(in cases where `opcache.enable_cli` is enabled, which can save some RAM, not sure why) and results in segmentation fault.
That is why this bundle generates a blacklist file which can be used for the `opcache.blacklist_filename` setting.
With this blacklist file, Opcache can still be enabled for CLI and there are no segmentation faults.
The generated file is stored under this path: `{$PROJECT_DIR}/var/cache/{$ENV}/swoole_bundle/opcache/blacklist.txt`

repository: pixelfederation/swoole-bundle

branch_name: refs/heads/develop

gha_language: PHP

repo_name: pixelfederation/swoole-bundle


text:
---
title:  Vision From the Volcano
date:  01/01/2019
---

**Evidence**: Rev. 1:3

The rocky landscape of Patmos, a volcanic island off the coast of modernday western Turkey, would experience its mightiest eruption in the form of a supernatural delivery of a message from the Son of God. In the Aegean Sea, on one of the northernmost islands of the Dodecanese (literally meaning “twelve islands”), the disciple at the heart of Jesus’ inner circle was in exile because of his unwavering commitment to the Lord. At the time, John was a figurative island on a literal island, separated from the other eleven apostles because he was the sole survivor.

Most historians believe Revelation was written in a.d. 95, near the end of Emperor Domitian’s rule. Unlike the emperors of his time who waited until after death to be hailed as gods, Domitian wanted his worship while he was alive.  John had walked with Jesus and wouldn’t deny Him to elevate an earthly emperor’s ego.

Roman historian Tacitus mentioned a Roman policy of exiling political prisoners to the small islands. At the time of banishment to Patmos, John was a very senior citizen, which would have made the terrain of this location difficult to navigate. Yet he knew God would see him through the trials (Rev. 1:9).

At the base of the mountain on Patmos John was separated from the familiar. However, halfway up the mountain lies the Cave of the Apocalypse, where it is believed that John resided when he received the visions. Today, it is a World Heritage site.

Fellow believers suffering persecution deeply desired hope, and Jesus would not fail to deliver a sure word. In fact, Revelation 1:3 gave immediate encouragement to John, who was enduring exile because he read, heard, and kept the Word of God. Verse three additionally initiates the first promise in a series of seven blessings, or “beatitudes,” occurring throughout Revelation.

In the book of Luke, the blind man desperately wanted his vision restored. Through the book of Revelation, Jesus maximized that singular opportunity to restore sight to invite the entire world to heed spiritually eye-opening instruction that would lead to permanent restoration to Him and the Father.

**React**

`1. What are some ways to decode biblical symbols?`

`2. Should any of Revelation’s language be taken literally?`

`3. Is Revelation’s mysterious message still relevant for Christians today?`

---

_Tamra Clarke, Weirsdale, Florida, USA_

repository: Adventech/sabbath-school-lessons

branch_name: refs/heads/stage

repo_name: Adventech/sabbath-school-lessons


text:
### [CVE-2021-41066](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-41066)
![](https://img.shields.io/static/v1?label=Product&message=n%2Fa&color=blue)
![](https://img.shields.io/static/v1?label=Version&message=n%2Fa&color=blue)
![](https://img.shields.io/static/v1?label=Vulnerability&message=n%2Fa&color=brighgreen)

### Description

An issue was discovered in Listary through 6. When Listary is configured as admin, Listary will not ask for permissions again if a user tries to access files on the system from Listary itself (it will bypass UAC protection; there is no privilege validation of the current user that runs via Listary).

### POC

#### Reference
- https://medium.com/@tomerp_77017/exploiting-listary-searching-your-way-to-system-privileges-8175af676c3e

#### Github
- https://github.com/ARPSyndicate/cvemon
- https://github.com/tomerpeled92/CVE

repository: trickest/cve

gha_language: HTML

repo_name: trickest/cve


text:
# Resources

Resources that we can leverage for our project. We will mainly use Python for machine learning.

## Data Sets
  * [The Met Open Data Set](https://console.cloud.google.com/marketplace/details/the-metropolitan-museum-of-art/the-met-public-domain-art-works?filter=solution-type:dataset&id=85a51186-8766-4379-a558-4d721e3a1190) Note that this dataset updates monthly.
  Also, check out:
   [Met Open Data Set GitHub](https://github.com/metmuseum/openaccess).
   [Met in Kaggle](https://www.kaggle.com/metmuseum/the-met)
   [GCP Article](https://cloud.google.com/blog/products/gcp/when-art-meets-big-data-analyzing-200000-items-from-the-met-collection-in-bigquery) on the Met Dataset.
   [Met API] (https://metmuseum.github.io/)

  * [Kaggle data sets](https://www.kaggle.com/datasets?sortBy=relevance&group=public&search=art&page=1&pageSize=20&size=sizeAll&filetype=fileTypeAll&license=licenseAll&fbclid=IwAR1d8cGImamMP_GqtUAP3ndsDmYjrq_XP_R04opfgaYiQqR1eqwljDlNxcM) This is probably not the best place to look for art datasets, but it's here if needed.

  * [Coding Durer Data Sources](http://codingdurer.de/data.html)

  * [Data.World Art Datasets](https://data.world/datasets/art) Contains links to many other datasets.

  * [MoMa Collection](https://github.com/MuseumofModernArt/collection).

  * [Behance Artistic Media Dataset](https://bam-dataset.org/) This site also contains image embedding.

  * [Tate Art Collection](https://www.tate.org.uk/about-us/digital/collection-data) Also, check out:
   [Microsoft Project](https://www.microsoft.com/inculture/arts/tate/), which uses this dataset.
   [Blog Post](http://museumtwo.blogspot.com/2013/11/visualizing-tates-collection-what-open.html)

  * [Walters Art Museum API](https://api.thewalters.org/)

  * [Cooper Hewitt API](https://collection.cooperhewitt.org/api/)

## Websites For Data
  * [Wiki Art](https://www.wikiart.org/)
  * [Wiki Media](https://commons.wikimedia.org/wiki/Main_Page)
  * [Open Art Data](https://www.openartdata.org/search/label/datasets)
  * [Victoria and Albert Museum](https://www.vam.ac.uk/api)

## Libraries
  * [OpenCV](https://opencv.org/)
  * [Keras](https://keras.io/) This is a framework (not really a library) built on top of TensorFlow.

## Processing Large Amounts of Data
  * [Apache Spark](https://spark.apache.org/) For distributive deep learning.
  * [Google Cloud Platform](https://cloud.google.com/) This is probably going to be VERY IMPORTANT.
  * [AWS](https://aws.amazon.com/)
  * [Microsoft Azure](https://azure.microsoft.com/en-us/)
  * [Graph QL](https://graphql.org/) Introduction to [Graph GL](https://medium.freecodecamp.org/a-beginners-guide-to-graphql-86f849ce1bec)

## Guides
  * [BigQuery](https://cloud.google.com/bigquery/what-is-bigquery?_ga=2.68514329.-987780976.1546634905)
  * [Deep Learning With OpenCV](https://www.pyimagesearch.com/2017/08/21/deep-learning-with-opencv/)
  * [Clustering Algorithms in Sklearn](https://scikit-learn.org/stable/modules/clustering.html)
  * [Assessment Metrics for Clustering Algorithms](https://medium.com/@ODSC/assessment-metrics-for-clustering-algorithms-4a902e00d92d)

## Related Research Papers
  * [A large scale database for computational painting categorization](https://www.researchgate.net/publication/272040931_Painting-91_A_large_scale_database_for_computational_painting_categorization)
  * [Unsupervised Learning of Artistic Styles with Archetypal Style Analysis](https://papers.nips.cc/paper/7893-unsupervised-learning-of-artistic-styles-with-archetypal-style-analysis.pdf?fbclid=IwAR3n9R16iiSD0Nav08bU3JbYSxNsC8zle_fjs3oo_H4IOv5YGhljaC6i6jI)
  * [BAM! The Behance Artistic Media Dataset for Recognition Beyond Photography](https://arxiv.org/pdf/1704.08614.pdf)
  * [Recognizing Art Style Automatically in painting with deep learning](https://www.lamsade.dauphine.fr/~bnegrevergne/webpage/documents/2017_rasta.pdf)

## Helpful Scripts
  * [Met Image Downloader](https://github.com/trevorfiez/The-Metropolitan-Museum-of-Art-Image-Downloader) This script basically allows us to scrape images from the Met. But note that the code here isn't that great.
  * [Computer-Vision-Basics-with-Python-Keras-and-OpenCV](https://github.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV)

## Questions to ask Google
  * How do you integrate BigQuery with Python?
  * How do we do machine learning with cloud platforms?


text:
---
title: Migrating Skins from RadMenu ASP.NET to ASP.NET AJAX
page_title: Migrating Skins from RadMenu ASP.NET to ASP.NET AJAX - RadMenu
description: Check our Web Forms article about Migrating Skins from RadMenu ASP.NET to ASP.NET AJAX.
slug: menu/appearance-and-styling/migrating-skins-from-radmenu-asp.net-to-asp.net-ajax
tags: migrating,skins,from,radmenu,asp.net,to,asp.net,ajax
published: True
position: 0
---

# Migrating Skins from RadMenu ASP.NET to ASP.NET AJAX

##

Since RadMenu for ASP.NET AJAX inherits its rendering from RadMenu for ASP.NET "Classic", it is rather easy to migrate a skin. The only change in the CSS class names is the control-specific prefix ("rm") and the capitalization of the first letter of the "Classic" class names afterwards. The following table lists all class names in the "Classic" version of RadMenu and their equivalents in the RadMenu for ASP.NET AJAX:

| RadMenu "Classic" class name | RadMenu for ASP.NET AJAX class name |
| ------ | ------ |
|link|rmLink|
|focused|rmFocused|
|expanded|rmExpanded|
|text|rmText|
|rootGroup|rmRootGroup|
|group|rmGroup|
|expandLeft|rmExpandLeft|
|expandRight|rmExpandRight|
|horizontal|rmHorizontal|
|vertical|rmVertical|
|item|rmItem|
|first|rmFirst|
|last|rmLast|
|topArrowDisabled|rmTopArrowDisabled|
|bottomArrowDisabled|rmBottomArrowDisabled|
|leftArrowDisabled|rmLeftArrowDisabled|
|rightArrowDisabled|rmRightArrowDisabled|
|topArrow|rmTopArrow|
|bottomArrow|rmBottomArrow|
|leftArrow|rmLeftArrow|
|rightArrow|rmRightArrow|
|disabled|rmDisabled|
|separator|rmSeparator|
|leftImage|rmLeftImage|
|slide|rmSlide|

repository: telerik/ajax-docs

gha_language: C#

repo_name: telerik/ajax-docs


text:
# Branch-Protections

Helps setup a master branch protection policy by setting:

- *Require pull request reviews before merging*: `true`
    - *Required approving reviews*: `1`
    - *Dismiss stale pull request approvals when new commits are pushed*: `true`
    - *Restrict who can dismiss pull request reviews*: your team list
- *Require status checks to pass before merging*: `true`
    - *Require branches to be up to date*: `true`
    - *Status checks required list*: any that succeeded in last 30 days
- *Include administrators*: `false`
- *Restrict who can push to matching branches*: your team list

(You can easily change these in the code)

## Setup

Provide a `.env` file in the root directory with the contents of `.env-sample`. They make the 'essential settings' and include your personal access token (repo level permission), team name for reviews, etc.

```javascript
GITHUB_API_TOKEN=82832askd9knsia42dbueiabdi2asndpasd1wwe
REVIEW_TEAM_NAME=dx-sdks-approver
YOUR_GITHUB_NAME=damieng
<EMAIL_ADDRESS>```

Do note the team name does not prefix the organization.

There are also additional but optional tweaks you can change

```javascript
// Optional tweaks - these are sensible defaults
const gitHubUrl = 'github.com'; // Change this if GitHub Enterprise
const dismissTeams = reviewTeams;
const copyChecksFromRef = [ 'master' ];
const daysPriorWithSuccessfulChecks = 30;
```

If you wish to change the settings simply change the object around line 50 to set the settings you want or do not want.

## Running

Run the script with the org/repo name, e.g.

```bash
node app.js @damieng/some-repo
```

Enjoy!

repository: damieng/github-scripting

gha_language: JavaScript

repo_name: damieng/github-scripting


text:
### InsertDateTime - A Plugin for Sublime Text v2 and v3

A Sublime Text plugin to insert date time stamps.

### Installation:

Manual installation only, this plugin has not been submitted to Package Control.

### In Brief:

    ST Command:      insert_date_time

    Optional Arg:    format: a date time formatting string (see below) which
                     controls which date time string should be inserted.

    Settings File:   InsertDateTime.sublime-settings

    Settings Fields:
                     formats: a list of date time formatting strings (see below)
                     which controls which date time strings will be displayed in
                     the overlay panel for user selection.

                     fixed_width_font: a boolean value which controls whether to
                     use a fixed width font in the overlay panel (default: true)

### Formats:

The date time formatting strings must be in the form used by the `strftime()` method of the Python DateTime class. For example `"%Y-%m-%d"` would give a date like `2016-06-17`, while `"%A %d %B %Y"` would give `Friday 17 June 2016`.

Lists of Python date time formatting characters can be viewed here: [Python.org strftime()](http://docs.python.org/3.3/library/time.html#time.strftime)

Note that `%Z` (time zone name) and `%z` (UTC offset) can not be used as these rely of information unavailable to a Sublime Text plugin. However the UTC offset value can be shown by the various standardized timestamps listed below.

The following standardized formatting strings may be used as the `format` arg or in the `formats` settings list:

    timestamp_iso_8601:        A timestamp in the following form:
                               2016-05-29T16:07:59+01:00
                               Used by: ISO 8601 and RFC 3339
                               This is the international and Internet standard.

    timestamp_rfc_3339:        Same as: timestamp_iso_8601
                               ISO 8601, the international standard, was
                               adapted from RFC 3339 the Internet standard.

    timestamp_rfc_3339_human:  A timestamp in the following form:
                               2016-05-29 16:07:59+01:00
                               Human readable form of: RFC 3339
                               Not valid under ISO 8601.

    timestamp_rfc_5322:        A timestamp in the following form:
                               Sun, 29 May 2016 16:07:59 +0100
                               Used by: RFC 5321, 5322
                               This is the standard used by email.

    timestamp_posix_time:      POSIX time, aka Unix or Epoch time; the
                               total number of elapsed seconds since:
                               1970-01-01 00:00:00 +00:00

### Usage:

The plugin can be launched by selecting `Insert Date Time` from the `Command Palette` or from key bindings that users must manually add by placing key bindings in their `Default (OS Name).sublime-keymap` file.

When the `InsertDateTime` plugin is run it first checks to see if the calling command included the optional `format` arg, if so the appropriate date time string will be inserted immediately.

If the command was not run with the `format` arg then the plugin will display the overlay panel with the date time strings that correspond to those set in the `formats` list in the settings file.

Note that the plugin works with multiple selections.

### Key Bindings Examples:

    Show the overlay panel with all the date time strings:
    { "keys": ["ctrl+?"], "command": "insert_date_time" },

    Insert the specified date time string:
    { "keys": ["ctrl+?"], "command": "insert_date_time", "args": {"format": "%Y-%m-%d"} },

    Insert the timestamp_iso_8601 date time string:
    { "keys": ["ctrl+?"], "command": "insert_date_time", "args": {"format": "timestamp_iso_8601"} },

### InsertDateTime.sublime-settings Example:

The default `InsertDateTime.sublime-settings` has 15 different date time stamps, users can overwrite this with their own file by placing a file called `InsertDateTime.sublime-settings` in their ST `User` config folder.

Users can view the default, and edit their user file, by clicking on the preferences menu:

    ST Menu --> Preferences --> Package Settings --> InsertDateTime

Here is an example `InsertDateTime.sublime-settings` file:

    {
        "formats":
        [
            "%H:%M:%S",
            "%Y-%m-%d",
            "%Y-%m-%d %H:%M",
            "%d-%m-%Y",
            "%d %B %Y",
            "%A %d %B %Y",
            "timestamp_iso_8601",
        ],
        "fixed_width_font": true
    }

### License

This package is licensed under The MIT License (MIT).

repository: mattst/InsertDateTime

repo_name: mattst/InsertDateTime


text:
# Product Engineer - Store

You will lead the technical and operational development of MUI Store.

## Details of the Role

- Location: Remote (preference for UTC-6 to UTC+3).
- Type of work: Full-time (contractor or employee [depending on circumstances](https://mui-org.notion.site/Hiring-FAQ-64763b756ae44c37b47b081f98915501))
- Start date: Immediately.
- Level: [4 or above](https://docs.google.com/spreadsheets/d/1dDdPD-flNXlgZ0E3ZxVvCDx27RFuhVWJrcfcjNu_I8k/edit#gid=0).
- We're a remote company, we prefer asynchronous communication over meetings.
- We work independently, the rest of us won't know what you're doing day-to-day unless you tell us.

## About the company

MUI started with Material-UI, the most successful React implementation of Google's Material Design.
It has gained a large following, not only due to the fidelity to Material Design, but also because of the number of components, its carefully designed component API, obsession for details, and community engagement.
Today, countless teams and organizations rely on our open-source libraries to build their design system.

A couple of years ago, we started to expand our suite of products.
We released [MUI X](/x/), a collection of advanced components; [MUI Design kits](/design-kits/), the MUI components available for the most popular design tools; and also host [Templates](/templates/), a set of pre-built UI kits.
We've seen rapid growth with all of them (2-3x per year) and we have more to come.

We work mainly asynchronously, relying mostly on written collaboration.
Every contributor has the freedom to define how they work — the rest of us won't know what you're doing day-to-day unless you tell us.
For additional details about the MUI team and culture, you can check our [careers](/careers/) and [about](/about/) pages and also our [public Handbook](https://mui-org.notion.site/Handbook-f086d47e10794d5e839aef9dc67f324b).

## Why we're hiring

The orders on the MUI Store are growing fast (x2-3 YoY).
We need talented people to keep that going!

The development and operations of the store are currently almost exclusively run by the executive team of the company (e.g. CEO). This team does no longer has enough bandwidth. The store doesn't receive the care it deserves. You will be responsible to step up and carry forward the development of the store on multiple fronts: implementing new features, reviewing new template submissions, defining the product direction, taking care of operational needs, and a lot more.

### Why this is interesting

You will get the opportunity to run MUI Store as a small start-up inside the company.

Our solution empowers React developers to build awesome applications – hundreds of thousands of developers use MUI every month.

## About the role

### What you'll do on a day-to-day basis

Depending on the day, you'll:

- Implement new features on https://mui.com/store/ to better support customers and contributors.
- Review new items submitted by the contributors to be hosted on the marketplace.
- Fix root problems raised by customers on the support channel.
- Own the operations: manage payouts, create sales reports, etc.

### Here are a few initiatives you might work on

Tasks that you might work on once you join:

- Revamp the fulfillment pipeline for MUI X Pro/Premium.
- Add a search page to scale the hosting to more items from contributors.
- Revamp the design of the store to match the new brand of mui.com.
- Migrate the store from Gatsby to Next.js. Gatsby was introduced at a point in time where it had more downloads than Next.js. Our CEO has extensive experience with Next.js since 2017 but wanted to ensure that MUI was supporting Gatsby well too.
  This requirement [is no longer relevant](https://npm-stat.com/charts.html?package=next&package=gatsby&from=2019-12-18&to=2021-12-18), and since the DX of Next.js is significantly better, time to migrate.
- Develop the SEO strategy, create new blog posts.
- Develop the SEA strategy, try retargeting, Google Ads, etc.

## About you

We're looking for someone with strong development skills.
More important than specific technical skills though is that you're a strong problem solver who loves to learn new things. Details matter to you.

### Skills you should have

- **Expertise in WordPress and the modern JavaScript ecosystem.**
- **A track record of demonstrating an eye for product and solving real-world user problems**. If you have a knack for solving problems at the root cause, finding boring solutions, we want you on our team.
- **Strong written and verbal communication skills**.
  As part of the team, you'll interface both directly and indirectly with community members and enterprise customers, and contribute to user documentation. Clear communication is fundamental in creating intuitive and compelling resources.
- **Ability to dive into complex problems**.
  You should be able to quickly assess, understand, and iterate upon aspects of our codebase.
- **Ready and willing to ask and answer questions**.
  If you're comfortable saying you're unsure, asking for help; but equally reaching out to assist others, you'll be an incredible addition to our team. We thrive because of continuous learning. First-time mistakes should be celebrated, not blamed.
- **Avoid monolithic deliverables**.
  You scope and stage your work into well-defined milestones to ship.

### What it would be nice if you had, but isn't required

- **Has or is pursuing a business side project**.  You were doing whatever was necessary to solve people's pain.

## Benefits & Compensation

Competitive compensation depending on the profile and location.
We are ready to pay top market rates for a person that can significantly push the mission forward.
You can find the other perks & benefits on the [careers](/careers/#perks-amp-benefits) page.

## How to apply?

[Apply now for this position 📮](https://airtable.com/shrdqo1Z6srZXGcvh?prefill_Applying+for=Product%20Engineer%20-%20Store&prefill_source=mui.com)

repository: oliviertassinari/material-ui

repo_name: oliviertassinari/material-ui


text:
[Lib Mgmt]

Library Management System in Java
=================================
> By Thinker & Performer

This repo contains the source code and documents related to
project `Lib Mgmt`.

Intro
=====
Lib-mgmt (short for *Library Management*) is a Java Web application wich provide a simple solution for small scale library to manage thier books and loan transactions.

Basically, it this project consists three components, the user and role management system, book management system and loan management. The user and role management system provide a layered role based user management interface. Users in this system will be divided into 4 types, the admin, librarians, students and guests. The book management system provides a interface to input new book entry, manage its copies, comment the book (also rate the comment), update and search the content. The third part, loan managemnt system is tightly integrated into the book management system, provide interface for librarian to lend book to student and accept book from student in different scenarios including book lost, broken, due and extending loan period.

Some popular frameworks and libraries are used in this project including Spring Framework MVC and Hibernate.

How to run
==========
 In order yo compile and run this project locally, genrally
speaking, you need following steps:

 1. Clone this project
 2. Update database connection configuration
 3. compile Java, Less code
 4. package libs, class files, static contents and configs
 5. deploy to application container

Prerequisite
============
 * JDK 1.8 +
 * MySQL 5.x
 * Tomcat 8.0 + / Jetty 8 +

repository: Jack-Q/Lib-Mgmt

repo_name: Jack-Q/Lib-Mgmt


text:
<h0>PHP Library</h0>

# Installation

We strongly encourage use of our official libraries for accessing the GoCardless API.

## Getting the library

Install from source:

	git clone git@github.com:gocardless/gocardless-legacy-php.git

or install from tarball:

	curl -L https://github.com/gocardless/gocardless-legacy-php/tarball/master | tar xzv

Check out our [PHP docs](/legacy/php) or view [source on Github](https://github.com/gocardless/gocardless-legacy-php).

### Community libraries

Please note: the GoCardless developer docs are not relevant when installing these libraries. GoCardless is also unable to provide support for any community built libraries.

#### Wordpress
* [GoCardless Wordpress plugin](https://github.com/DHS/wp-gocardless) by [David Haywood-Smith](https://twitter.com/DHS)
* [GoCardless for Wordpress](http://codecanyon.net/item/gocardless-for-wordpress-plugin/3207246?ref=outerbridge) by [Outerbridge](http://outerbridge.co.uk/2013/03/gocardless-wordpress-plugin/)

#### ClientExec
* [clientexec-gocardless](https://github.com/peterlaws/clientexec-gocardless) by [peterlaws](https://github.com/peterlaws/clientexec-gocardless)

repository: joshkehn/api-docs

repo_name: joshkehn/api-docs


text:
# CoreLib-Kotlin

[![](https://jitpack.io/v/egorikftp/CoreLib-Kotlin.svg)](https://jitpack.io/#egorikftp/CoreLib-Kotlin)

I don't like write the same code every time :sunglasses:

Let's save your time and use modern features Kotlin language :heartpulse:

Two simple steps:

1. Add to project build.gradle file this code:

```groovy

allprojects {
	repositories {
		...
		maven { url 'https://jitpack.io' }
	}
}
```

2. Add dependency for you app build.gradle:

```groovy
  dependencies {
  	implementation 'com.github.egorikftp:CoreLib-Kotlin:0.0.7'
}
```

repository: egorikftp/CoreLib-Kotlin

gha_language: Kotlin

repo_name: egorikftp/CoreLib-Kotlin


text:
# CVE-2017-15944

CVE-2017-15944 Module for [Router Scan Project](https://github.com/yukar1z0e/RouterScan)

## How To Use

prepare

```bash
pip3 install ipcalc,requests
```

usage

```bash
python3 scanner.py
```

## Copyright

some part of this repository is inspired by [paper.seebug.org](https://paper.seebug.org/540/) with some changes for APIs of [Router Scan Project](https://github.com/yukar1z0e/RouterScan).


text:
---
layout: default
---

# 1996-12-04 - Re:

## Header Data

From: "Matthew J. Miszewski"<EMAIL_ADDRESS>To: Bovine Remailer<EMAIL_ADDRESS>Message Hash: ef84b634bc196e0d06027c7df268a9314b438653a71f557652c630a153a1866d

Message ID<EMAIL_ADDRESS>Reply To: _N/A_

UTC Datetime: 1996-12-04 04:20:11 UTC

Raw Date: Tue, 3 Dec 1996 20:20:11 -0800 (PST)

## Raw message

```
{% raw  %}From: "Matthew J. Miszewski"<EMAIL_ADDRESS>Date: Tue, 3 Dec 1996 20:20:11 -0800 (PST)
To: Bovine Remailer<EMAIL_ADDRESS>Subject: Re:
Message-ID<EMAIL_ADDRESS>MIME-Version: 1.0
Content-Type: text/plain

At 01:45 PM 12/3/96 EST, Bovine Remailer wrote:
>At 10:50 AM 12/3/1996, Matthew J. Miszewski wrote:
>>Take, for example, the practice of redlining.  How are people who live in
>>"bad" neighborhoods supposed to not reveal that information.
>
>You may lend your own money to whomever you wish.  If you do not wish
>to lend money to somebody, that is your business.

I consider it my business also, when people are denied opportunity because
of where they live.  I may also hire whomever I wish, but I would have to
pay the consequences if I happened to discriminate based on a protected
class while doing so.  That is the society in which I live.  If I dont like
it, I try to change it.  Our society is not libertarian.  If I would prefer
that form of society I would have to accept that result.  Therefore I
choose a balance between liberty and social justice.  There are times when
government should intervene.  I believe it should be as infrequent as
possible, but would not want to live in a society where disinfranchised
people have no possible recourse.  Your choice would apparently be different.

>It is difficult to understand why redlining should be illegal, to the
>extent that it even occurs.  When it does occur, we expect that eager
>entrepreneurs such as yourself will rush in to grab new customers.

I have not heard serious doubts for a while that redlining occurs.  But
then again we are on the net.  I would love to have the financial
wherewithal to startup such an enterprise.  Unfortuantely I reside in one
such neighborhood.  It is difficult enough to raise money to run a small
business (and turned out to be much easier to do without any bank lending
at all).  I have talked with people about starting their own banks.  When
you are working to make sure all the bills are paid it is a bit difficult
to also build an entirely new socio-economic structure.

>As for the privacy issue, you seem to be proposing that you have some
>sort of right to borrow money on terms which are not acceptable to the
>lender.  You need not give your address just as you need not borrow
>the money.

Actually I was stating the opposite.  You do not need to do so.  You just
wont get the money if you dont.

>Of course, we hardly live in a free banking era.  Most people would
>prefer to bank with a company that respects their privacy.  Yet, banks
>are so tightly controlled in the United States that they most often
>will not dare to protect the privacy of their customers for fear of
>regulatory consequences.  When the service is provided, it cannot be
>advertised.

Whom would the service be more readily available to?  Who uses tax-havens?
Who has access to swiss bank accounts?  Are you insinuating that my local
bank actually has anonymous accounts and just won't tell me?  I wish that
people DID value anonymous banking in this country.  The fact is they just
don't care.  As long as they get short lines or myriads of ATM machines
they are happy.  I am largely in favor of banking deregulation.  There are
places where I simply draw the line.  Utter racism is one of them.
Everyone can now clamor that it just isnt true.  Banks have never
discriminated.    Whatever.

(snip)
>You are in the unpleasant position of appealing for protection from
>the very people who have robbed you of your privacy.

You are right.  It is far from a perfect system.  We make trade-offs every
day.   The real world I live in is just not as simple as the Libertarian
Wet Dream(TM).

>
>Red Rackham
>

Matt

{% endraw %}
```

## Thread

+ Return to [December 1996](/archive/1996/12)

+ Return to "["Matthew J. Miszewski" <mjmiski@execpc.com>](/authors/matthew_j_miszewski_mjmiski_at_execpc_com_)"

+ 1996-12-04 (Tue, 3 Dec 1996 20:20:11 -0800 (PST)) - Re: - _"Matthew J. Miszewski"<EMAIL_ADDRESS>


text:
# Opensource JobScheduler (http://www.sos-berlin.com)

Includes:

* JobScheduler master
* JobScheduler JOC

# General

* Ports: 4444 (web interface), 4446 (JOC), 40444 (agents),
* mysql environment will be used in startup script

# Build

```
docker-compose up

```

repository: ericogr/jobscheduler-docker

repo_name: ericogr/jobscheduler-docker