text:
{
  "title": "The urban footprint of the largest urban areas of Brazil https://t.co/86N26TTaeV Map created in R using open data from #geobr package https://t.co/2ePH5YAHFg #rstats #rspatial https://t.co/hBxmBY0NcW",
  "date": "2019-10-31"
}

# rstats

> **Statisticians and Data Scientists** (@statsanddatasci; 81/16): The urban footprint of the largest urban areas of Brazil https://t.co/86N26TTaeV
>
Map created in R using open data from #geobr package https://t.co/2ePH5YAHFg #rstats #rspatial https://t.co/hBxmBY0NcW  [↪](https://twitter.com/dataandme/status/1189640816995770368)

<!-- -->

> **Danielle Navarro** (@djnavarro; 70/6): #TidyTuesday Week 2019/44 🐿 Squirrel Sightings in NYC's Central Park - quick maps of the density and small multiples of each for color type.
>
Article about the NYC Squirrel Census: https://t.co/bvcD8NkI4g
>
#r4ds #ggplot2 #tidyverse #rstats #dataviz https://t.co/HW0sqMw9QV  [↪](https://twitter.com/dataandme/status/1189643662222938114)

<!-- -->

> **Muhammad Aswan Syahputra** (@aswansyahputra_; 19/6): I am a huge #rstats user but don't use GitHub very often. But, I needed to teach my students how to use it to collaborate on projects. My material must be ok since ~90 of my students were using it by the end of my classes yesterday. Repo is here: https://t.co/jhBzKibKs6  [↪](https://twitter.com/dataandme/status/1189608845968138241)

<!-- -->

> **mixOmics** (@mixOmics_team; 18/4): Try to use -> (not <-)  for #rstats dplyr assignments!
1. It matches how we think:
input %>% transformed %>% more transformed -> output
2. It helps running code in the console as you develop it: you always need to run the beginning of a chunk and not assigning shows what you get  [↪](https://twitter.com/dataandme/status/1189632440685551616)

<!-- -->

> **CRAN Package Updates** (@CRANberriesFeed; 17/3): I’ve had a few recruiters and hiring managers telling me how much they appreciated a personal website which contains all my projects, talks, codes, publications, blogs, videos, Shiny apps, and links to LinkedIn/GitHub/Twitter so they don't get lost from zillions of links. #rstats https://t.co/pNEwQad00R  [↪](https://twitter.com/dataandme/status/1189626524996997120)

<!-- -->

> **tidyverse tweets** (@tidyversetweets; 17/2): Y'all, I made an awesome @accidental_aRt!  (I built my own style guide with bourbon & right assignment) https://t.co/Q0xNLVSacv  [↪](https://twitter.com/dataandme/status/1189648016472444929)

<!-- -->

> **CRAN Package Updates** (@CRANberriesFeed; 17/1): Found this "New to Me" SQL + #Rstats via {dplyr} resource from @datacarpentry last night - h/t to @RikaGorn for the discussion.
>
Great resource for incorporating and practicing SQL queries written automatically by {dplyr}
>
https://t.co/He8mvDjSoM
>
#tidyverse  [↪](https://twitter.com/dataandme/status/1189625118311538692)

<!-- -->

> **Matt.0** (@MattOldach; 16/3): Folks! Do I know four people willing to push me over the edge?! #youtube #selfpromote #stats
>
New NLP lectures starting this week! #rstats and #python https://t.co/v436FlugkG  [↪](https://twitter.com/dataandme/status/1189671188274065408)

<!-- -->

> **Dr. GP Pulipaka** (@gp_pulipaka; 14/13): Download This Free eBook (1962 Page-PDF): #Algebra, Differential #Calculus For #MachineLearning. #BigData #DataScience #AI #IoT #IIoT #Python #RStats #JavaScript #ReactJS #GoLang #CloudComputing #Serverless #DataScientist #Linux #Mathematics #Books
https://t.co/QMNldkuE0a https://t.co/hReQ7emj0f  [↪](https://twitter.com/dataandme/status/1189696705962266624)

<!-- -->

> **Los Angeles R Users** (@la_Rusers; 13/19): Quick analysis using #Pandas, #Matplotlib, and #Plotly. #BigData #Analytics #DataScience #AI #MachineLearning #IoT #IIoT #PyTorch #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #GoLang #CloudComputing #Serverless #DataScientist #Linux
https://t.co/3FDXym6NQa https://t.co/CNhosbjWP6  [↪](https://twitter.com/dataandme/status/1189684296912523264)

<!-- -->

> **Venkatesh-Prasad Ranganath** (@orderwithchaos; 13/6): last week i prayed this dataset would get picked for #TidyTuesday. nice to know @thomas_mock & i are on the same page.
>
code: https://t.co/29TX9Lw1J4
>
#rstats #ggplot2 #gganimate #squirrels https://t.co/vIMqYgIayP  [↪](https://twitter.com/dataandme/status/1189658282618699776)

<!-- -->

> **Samuel Brown** (@DrSamuelBrown; 13/4): Getting prepped to teach this👇to a group of undergraduates this afternoon, most of whom will have never seen #rstats before.
>
Thank goodness for @rstudio & https://t.co/WvmRrDyFM7
>
🌱🌱🌱🌱🌱🌱
💻💻💻💻💻💻
>
Background: https://t.co/I85QBpw8BL
Rmd & data: https://t.co/vyovFmDUkJ https://t.co/t4lGZPvQUk  [↪](https://twitter.com/dataandme/status/1189627903241146368)

<!-- -->

> **Hlynur Hallgrímsson** (@hlynur; 10/1): 📢My #rstats friends, 100% of data analyst/data scientist job applications ask you for a personal portfolio website. This is the LAST chance to learn how to build one by the end of 2019.
>
🗓Mark your calendars & please spread the word
>
Los Angeles East: Nov 6⃣
Irvine: Nov 1⃣6⃣ https://t.co/Ort7LuJ4vn  [↪](https://twitter.com/dataandme/status/1189621311019966464)

<!-- -->

> **Dr. Erin Buchanan** (@aggieerin; 9/11): What is NLP & What Do NLP Scientists Do? #BigData #Analytics #DataScience #AI #MachineLearning #NLProc #IoT #IIoT #PyTorch #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #GoLang #CloudComputing #Serverless #DataScientist #Linux
https://t.co/PbGj6otEha https://t.co/upADR5thyl  [↪](https://twitter.com/dataandme/status/1189708847838982144)

<!-- -->

> **Ektropos** (@Ektropos; 9/2): Y’all @ryanlinnbrown just taught her first R workshop to a group of undergrads!! 😍 #statskween #rstats #rladies https://t.co/XvU9brz4Sr  [↪](https://twitter.com/dataandme/status/1189663160455880705)

<!-- -->

> **Henrik Bengtsson** (@henrikbengtsson; 7/2): {1b*} Sharpened version of the dot density map of Bosnia and Herzegovina with additional info on dot size
>
#BosniaHerzegovina #BIH #RepublikaSrpska #census #demography #population #maps #rstats #dataviz https://t.co/tbapr6b4mn  [↪](https://twitter.com/dataandme/status/1189611277347409921)

<!-- -->

> **Nicolyn Charlot** (@nhcharlot; 7/1): As a longtime and expert #rstats user it's amusing to be on the other side of "OMG why in the hell is this designed this way" feelings as I work my way through pandas.  [↪](https://twitter.com/dataandme/status/1189650255173709824)

<!-- -->

> **Michael Reed** (@50_first_data; 6/8): Free eBook:The #DataEngineering Cookbook.  #BigData #Analytics #NoSQL #DataScience #AI #MachineLearning #IoT #IIoT #PyTorch #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #GoLang #CloudComputing #Serverless #DataScientist #Linux #Books
https://t.co/FnlRLpNShL https://t.co/c3SkII4wdC  [↪](https://twitter.com/dataandme/status/1189690293504770053)

<!-- -->

> **Robert Grant** (@robertstats; 6/2): #rstats friends, come learn some basics of building a #Shiny app in R with a Halloween-themed dataset at tonight's meetup with @RLadiesSB. https://t.co/yRiR2iaLys  [↪](https://twitter.com/dataandme/status/1189647646560071680)

<!-- -->

> **Gráficos y Métricas** (@gmbeisbol; 6/1): Have a look at this insightful #dataviz. Whenever things naturally (or even with some stretch) fall into three parts, there is some work for {tricolore} 📦 #rstats
Thanks to His Triangular Majesty @jschoeley https://t.co/hoXAWfOUjv  [↪](https://twitter.com/dataandme/status/1189622942159777793)

<!-- -->

> **VOSON Lab ANU** (@vosonlab; 5/5): BEST #Redis Course on Udemy
>
Learn Redis And Utilize Jedis With Spring Data Redis
>
👉 https://t.co/DPxvtk0ne3
>
#NoSQL #Developer #PHP #MongoDB #BigData #Analytics #DataScience #Java #JavaScript #ReactJS #Golang #RStats #RProc #CloudComputing #Serverless #Linux #DataScientist  [↪](https://twitter.com/dataandme/status/1189649509623582720)

<!-- -->

> **Los Angeles R Users** (@la_Rusers; 5/4): Always good to be reminded: #R #rstats #AcademicChatter
> citation()
>
To cite R in publications use:
>
R Core Team (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://t.co/n1s5ywx2Da.  [↪](https://twitter.com/dataandme/status/1189704309820317696)

<!-- -->

> **Shantanu Mullick** (@ShantanuMullick; 5/3): setwd() 🙃 #rstats https://t.co/5xNHQrwm0q  [↪](https://twitter.com/dataandme/status/1189618994338291718)

<!-- -->

> **Alex Bresler** (@abresler; 5/2): #rstats friends in Orange County, we will have a 90-min workshop @UCIrvine providing hands-on instructions on how to build-out a data analyst/data scientist website using @RStudio and @Netlify. Space is limited so get your **FREE** ticket here! https://t.co/CoGpMXUbMY https://t.co/2LFZERaGw8  [↪](https://twitter.com/dataandme/status/1189617633798193152)

<!-- -->

> **Meg** (@Meganwatts14; 5/1): This week #TidyTuesday, I look at the combination of squirrels' fur color, their location in Central Park and their activity by age and shift.
>
Code : https://t.co/3JtF873OmO
>
#rstats #dataviz https://t.co/gLaIh4gNyj  [↪](https://twitter.com/dataandme/status/1189666127942733824)

<!-- -->

> **Nick Barrowman** (@nbarrowman; 5/1): For my #tidytuesday this week, I counted the number of squirrel observations per hectare and generated a heat-map of the results. You can see a geographical affect on the observations! #rstats
>
Code is available at: https://t.co/Z2HVKASqph https://t.co/QYi387a0x7  [↪](https://twitter.com/dataandme/status/1189648726144339968)

<!-- -->

> **Dr. GP Pulipaka** (@gp_pulipaka; 4/11): Essential AI Books. #BigData #Analytics #DataScience #AI #MachineLearning #ReinforcementLearning #IoT #IIoT #PyTorch #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #GoLang #CloudComputing #Serverless #DataScientist #Linux #Books
https://t.co/NQWLHMAXYv https://t.co/c0WnLiCDyu  [↪](https://twitter.com/dataandme/status/1189686324317474817)

<!-- -->

> **Pablo Cabrera** (@pablocalv; 4/5): Episode 10: Horror Movies is available at https://t.co/CMSQyQCzNu and in your feed! #dataviz this week by @committedtotape and @ewBarba. Thanks to @sastoudt for pointing us to the new SQUIRRELS dataset! #TidyTuesday #rstats  [↪](https://twitter.com/dataandme/status/1189674046256033792)

<!-- -->

> **IAM Platform** (@IAM__Network; 4/3): The Chaos Game: an experiment about fractals, recursivity and creative coding | Fronkonstin https://t.co/wDmlEZeRzo #rstats #generativeArt #maths  [↪](https://twitter.com/dataandme/status/1189662873997512704)

<!-- -->

> **mohammad rosidi** (@rosidi2610; 4/1): "Yes, we CRAN!" Best conference title ever :-D Abstract submissions due tomorrow. https://t.co/SX41s4CYHL #rstats  #rstudio  [↪](https://twitter.com/dataandme/status/1189666969173164032)

<!-- -->

> **Alex Hallam** (@alexhallam6_28; 4/1): Has anyone used the rtweets 📦 with a premium twitter #API subscription? I called the api with search_fullarchive() function once, got 9k obs, and somehow used all my 100 requests...how are requests calculated?! tips?.... #rstats #RLadies #HumpDay #SentimentAnalysis #SexualHealth https://t.co/hxjrRhtOJv  [↪](https://twitter.com/dataandme/status/1189666458801803265)

<!-- -->

> **Dr. GP Pulipaka** (@gp_pulipaka; 4/1): When you’re running a piece of code that takes forever, and you really have no idea if it’s even doing what you want. #rstats https://t.co/fsq6LI1y6J  [↪](https://twitter.com/dataandme/status/1189664069927800832)

<!-- -->

> **👻 cjlortie** (@cjlortie; 4/1): Flexing my💪because simple functions I wrote two months ago are helping me churn out a ton of analyses on deadline. Rock and roll, #rstats.  [↪](https://twitter.com/dataandme/status/1189658970002272256)

<!-- -->

> **Devin M. Gill** (@DmGill_CogNeuro; 4/1): HelloWorld <- genBaby(sex = ‘f’, niter=4, parentsleep=0, parenthappy=10) #rstats #babies #buymeacoffee https://t.co/WiDheiFIEW  [↪](https://twitter.com/dataandme/status/1189647374626500608)

<!-- -->

> **Dr. GP Pulipaka** (@gp_pulipaka; 3/5): New version of RCy3 in @Bioconductor  includes new functions, new options and better performance for scripting @cytoscape (https://t.co/clnvMQGDow). Also checkout our growing collection of #Rstats  notebooks (https://t.co/vePZ5ThRmh) and demo videos (https://t.co/DUUl8zblLB). https://t.co/1Luxb4W0WT  [↪](https://twitter.com/dataandme/status/1189688889109168129)

<!-- -->

> **Ellis Hughes** (@ellis_hughes; 3/1): @dataandme @thomasp85 @hadleywickham @JennyBryan Did you all get put on the big-screen as #rstats celebrities?!? @topepos , @JennyBryan, @hadleywickham , @thomasp85 ,  and @dataandme in one place 🥇  [↪](https://twitter.com/dataandme/status/1189690068883169281)

<!-- -->

> **tidyverse tweets** (@tidyversetweets; 3/1): A great hands-on #WMUevalcafe today working in R thanks to @rborashko!! #WMU #rstats https://t.co/yj9eW7qKeC  [↪](https://twitter.com/dataandme/status/1189633091981045760)

<!-- -->

> **Pakin** (@pakinproton; 3/1): Yay #Data! My speciality. I love analyzing large datasets wirh #rstats in @rstudio so once I have enough client interactions detailed in a spreadsheet or report, I play around to look for patterns that can direct my future focus. #SproutChat https://t.co/FbaMhlVrKE  [↪](https://twitter.com/dataandme/status/1189632006470230016)

<!-- -->

> **Jessica Streeter** (@phillynerd; 3/1): 🔶Expert Your Spotlight🔶
>
Get the latest edition of Brett Lantz's popular Books Solve real-world #Data problems with R and #MachineLearning 'Machine Learning with R, 3rd Edition'
>
👉 https://t.co/uF5ksXSack
>
#DataScience #RStats #Python #Algorithm #Programming #DataScientist  [↪](https://twitter.com/dataandme/status/1189614524501901312)

<!-- -->

> **tidyverse tweets** (@tidyversetweets; 3/1): Dejo aquí el código y los datos para replicar el #ggplot2 #rstats
>
https://t.co/FqB1Zmlk0q  [↪](https://twitter.com/dataandme/status/1189608779370946560)

<!-- -->

> **tidyverse tweets** (@tidyversetweets; 2/3): Want to make use of AWS resources from #rstats? Dyfan Jones has you covered with this post.
>
#DataScience https://t.co/4cmUhieNhe … https://t.co/dAffMLn4g6 #Analytics thanks @RichardEudes  [↪](https://twitter.com/dataandme/status/1189635937640308736)

<!-- -->

> **Travis Gerke** (@travisgerke; 2/2): #Rstats Question: do you know of a package or function that creates a new column with the same row values for a dataset? Ex: I want Ind_1 for all rows in a new column. I’m trying to stick with the #Tidyverse, but I’ll use anything that works! @MorphoFun @UrFreundHannah  [↪](https://twitter.com/dataandme/status/1189697254547804161)

<!-- -->

> **Christophe Nicault** (@cnicault; 2/2): Tried a #rstats 3D anaglyph with #rspatial using a @coolbutuseless 📦, then a very well-written (but baffling/deprecated) attempt by @BrodieGaslam May just result to using photoshopping 🙄🤷♂ https://t.co/i2he6DWCqH  [↪](https://twitter.com/dataandme/status/1189691788208021506)

<!-- -->

> **EvalCenterWMU** (@EvalCenterWMU; 2/2): The DCR conference @rstatsdc is next week! This is the second year we have it and the speakers are awesome.  I am speaking about using APIs and managing your cloud with #rstats. Join us! #rstatsdc #datadc tix - https://t.co/SigrBOVWoK  [↪](https://twitter.com/dataandme/status/1189678887434895360)

<!-- -->

> **Ilya Kashnitsky** (@ikashnitsky; 2/1): My first r packages have been released on GitHub.  The edtaR package is a package of sample dataset sets in the environmental field sourced from open data provided by USGS.
>
#analytics #rstats #dataanalytics
>
https://t.co/TqHb1g2l8R https://t.co/7jn0d9NaWT  [↪](https://twitter.com/dataandme/status/1189703919787593729)

<!-- -->

> **TidyTuesday Podcast** (@tidypod; 2/1): I just released a cleaner version of cfsalesdata1. https://t.co/TGyQ6NBcel . This dataset is a great way to test out various retail models which include holidays, promotional events, and hierarchical data.  #rstats https://t.co/bJ4wa7FIHx  [↪](https://twitter.com/dataandme/status/1189687135500148737)

<!-- -->

> **IAM Platform** (@IAM__Network; 2/1): @yodacomplex thank you for the #OpenAccess #Rstats intro you’ve provided! Big help for @theorecol’s course  [↪](https://twitter.com/dataandme/status/1189663669854052364)

<!-- -->

> **Data Scientists** (@DataScientistsF; 2/1): Rcpp, Camarón de la Isla and the Beauty of Maths | Fronkonstin https://t.co/WDUo4UY7fM #rstats #maths #generativeArt  [↪](https://twitter.com/dataandme/status/1189663195524403200)

<!-- -->

> **David Hood** (@Thoughtfulnz; 2/1): This query should not be taking this long. #sql #rstats https://t.co/F2580Cokkc  [↪](https://twitter.com/dataandme/status/1189618617798709248)

<!-- -->

> **Dr Gavin Simpson** (@ucfagls; 1/4): A #JavaScript / #Python / PHP #Cryptocurrency Trading API. #BigData #Analytics #DataScience #FinTech #BlockChain #IoT #IIoT #PyTorch #RStats #TensorFlow #Java #ReactJS #GoLang #CloudComputing #Serverless #DataScientist #Linux
https://t.co/QJii19AOkF https://t.co/N2UQ1YCgCN  [↪](https://twitter.com/dataandme/status/1189713249639399424)

<!-- -->

> **joranelias** (@joranelias; 1/4): IAM Platform Curated Retweet:
>
Via: https://t.co/qx1bkvOcsh
>
Free eBook:The #DataEngineering Cookbook.  #BigData #Analytics #NoSQL #DataScience #AI #MachineLearning #IoT #IIoT #PyTorch #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #GoLang #Cl  [↪](https://twitter.com/dataandme/status/1189698377606402051)

<!-- -->

> **Dr. GP Pulipaka** (@gp_pulipaka; 1/4): IAM Platform Curated Retweet:
>
Via: https://t.co/qx1bkvOcsh
>
What is NLP &amp; What Do NLP Scientists Do? #BigData #Analytics #DataScience #AI #MachineLearning #NLProc #IoT #IIoT #PyTorch #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #GoLang  [↪](https://twitter.com/dataandme/status/1189709018517966848)

<!-- -->

> **tidyverse tweets** (@tidyversetweets; 1/3): Want to make use of AWS resources from #rstats? Dyfan Jones has you covered with this post.
>
#DataScience https://t.co/WQ9DdXMn8V … - https://t.co/rBE6bukVxL #DataScience thanks @RichardEudes  [↪](https://twitter.com/dataandme/status/1189635720794775554)

<!-- -->

> **Free Coupon Discount** (@NewCoupon2; 1/2): Training a Variational Autoencoder with generative moment matching neural networks on Fashion MNIST dataset using #Keras / #Tensorflow and gnn R #rstats #MachineLearning  package. Pretty fast training and realistic results. https://t.co/YoNcDxBrZI  [↪](https://twitter.com/dataandme/status/1189703263345623040)

<!-- -->

> **Antonio S. Chinchón** (@aschinchon; 1/2): It just occurred to me that I can use case_when() within filter() ... I could probably even use it in summarize() #rstats #tidyverse  [↪](https://twitter.com/dataandme/status/1189664926123646979)

<!-- -->

> **Gerry C.** (@g_casteleiro; 1/2): spent some time on these #rstats #rmarkdown #YAML for better knitting to PDF (i.e with toc without 'Contents' header)
https://t.co/kJv4irLUGp  [↪](https://twitter.com/dataandme/status/1189641949260767234)

<!-- -->

> **Scott Davis** (@scottyd22; 1/2): @sckottie worked with #rorcid this week to explore an #rstats #rmarkdown capacity to have a reproducible #CV - would love input - feels clunky still. @gordonblasco
>
https://t.co/LSI4yLgNg9  [↪](https://twitter.com/dataandme/status/1189647123945529345)

<!-- -->

> **V E N U 👨🏽💻** (@itsvenu_; 1/2): TIL about Morpheus.  Something like #Rstats and #pandas on top of JVM while leveraging parallelism.  https://t.co/Q4MZqYC9G6 #datascience #Java #data #programming  [↪](https://twitter.com/dataandme/status/1189623528917733379)

<!-- -->

> **JD Long** (@CMastication; 1/1): Error: Package "DescTools" not available for R Version 3.4.2 #tidyverse #rstats https://t.co/hKhtrpyxch  [↪](https://twitter.com/dataandme/status/1189620542917304320)

<!-- -->

> **Thomas Mock** (@thomas_mock; 1/1): Rstudio code chunks #tidyverse #rstats https://t.co/iW6XxdzhRQ  [↪](https://twitter.com/dataandme/status/1189620544422993923)

<!-- -->

> **Megan_Crawford** (@Scenario_Megan; 1/1): I retweeted a breathing timing gif yesterday, but I didn't personally like the timing much, so I have writing the code for "make your own breathing gif" in #rstats with #gganimate
>
https://t.co/jSMLpN5Yua
>
Just set the seconds at the top to take it at the appropriate pace https://t.co/9lBU9Tj0if  [↪](https://twitter.com/dataandme/status/1189698395360784384)

<!-- -->

> **OC RUG** (@oc_rug; 1/1): There should be an R package that tests all your assumptions. #rstats  [↪](https://twitter.com/dataandme/status/1189692084795650049)

<!-- -->

> **Antonio S. Chinchón** (@aschinchon; 1/1): Pre #ASNAC2019 conference workshop: VOSON Lab Tools for Collecting and Analysing Online Networks. 27 November 2019, 3-4:30 pm at @Flinders, Adelaide. @RobAckland @ANSNA_Au #SNA #rstats #VOSONDash #vosonSML https://t.co/CWf665AOGF https://t.co/ATM08gJwZh  [↪](https://twitter.com/dataandme/status/1189686969527226368)

<!-- -->

> **James Custer** (@Jamesguy_Inc; 1/1): If you're at #RMAIR, come to Interlaken tomorrow at 11 AM to learn about "Multiple Approaches to Assessing the Impacts of Student Success Initiatives: Three Examples at UNLV"!  We'll be discussing various models we use in #rstats and why we like them (or don't)!  [↪](https://twitter.com/dataandme/status/1189681263067590656)

<!-- -->

> **Rachel Lesniak** (@ralesniak; 1/1): @JulieMOPetersen I’m a bit late seeing this, but you might be interested in my vtree #rstats package. See https://t.co/w1xCjbHPms  [↪](https://twitter.com/dataandme/status/1189680771180703745)

<!-- -->

> **Matt.0** (@MattOldach; 1/1): CRAN updates: xaringan https://t.co/y5W2NTKSXT #rstats  [↪](https://twitter.com/dataandme/status/1189618503986360320)

<!-- -->

> **Buzz Techy** (@BuzzTechy; 1/1): Our #bioconductor package is released today. Feedback/suggestions will be appreciated.
>
More features will be added in future!
>
#rstats #bioinformatics #epigenomics
>
https://t.co/fG1CyPqtcd https://t.co/8KA9K7bRHp  [↪](https://twitter.com/dataandme/status/1189678824897822723)

<!-- -->

> **Miles Valencia** (@MilesToKilo; 1/1): @tmorris_mrc I'll send you one of my hand-made graphics::plot pentagons. Guaranteed to make #rstats devotees scowl.  [↪](https://twitter.com/dataandme/status/1189672282576367618)

<!-- -->

> **joranelias** (@joranelias; 1/1): Wait, I just did a Ctrl+Alt+R to a run of some very computationally heavy #rstats code in an #Rmd, assuming THAT would be cached since I had cache=TRUE for the chunks in question. Now when I #blogdown::serve_site I get the old results. Is there anything I can do to not run it x2?  [↪](https://twitter.com/dataandme/status/1189664772033253376)

<!-- -->

> **Luke Smith** (@lksmth; 1/1): Used @bball_ref, pulled some basic stats (FT, FTA, FTP, ORB, DRB,  AST, STL, BLK, TOV, PF) from 18-19 season. TOV and DRB were sig. predictors of win%  after accounting for shared variance. GSW was an outlier. Had fewest DRB but high win %.  #NBA #rstats https://t.co/LmxydW3bkc https://t.co/RaV3Mwwjjm  [↪](https://twitter.com/dataandme/status/1189661410168471552)

<!-- -->

> **👻 cjlortie** (@cjlortie; 1/1): #rstats workshop @univmiami.  Is aggregate() the best kept base R secret? 😅 https://t.co/pyLcHsopH7  [↪](https://twitter.com/dataandme/status/1189659243865083905)

<!-- -->

> **tidyverse tweets** (@tidyversetweets; 1/1): Interesting use cases for the pins 📦 Excited to check it out #rstats  https://t.co/GRMUEWEUHi  [↪](https://twitter.com/dataandme/status/1189651590199808001)

<!-- -->

> **CRAN Package Updates** (@CRANberriesFeed; 1/1): Why waste your time reading this ?
#WhoCares #rstats https://t.co/KQEQbclrTi  [↪](https://twitter.com/dataandme/status/1189648299067920388)

<!-- -->

> **Trina** (@trina_cc; 1/1): @csmarcum My very favourite thing about the #rstats community  [↪](https://twitter.com/dataandme/status/1189637226348498944)

<!-- -->

> **Keep It Surly** (@surlyurbanist; 1/1): When I began with R, loading large files was a nightmare, now I can't stop using #feather and my life is easier #rstats  [↪](https://twitter.com/dataandme/status/1189630113824018434)

<!-- -->

> **Naomi** (@NaomiJBall; 1/1): #rdata #rstats: Data set for Rahul Dravid - https://t.co/yDEb7vghAg  [↪](https://twitter.com/dataandme/status/1189618824313757696)

<!-- -->

> **Shiloh Bradley** (@ShilohRBradley; 0/3): #Rstats #Python #JuliaLang #CienciaDeDatos #InteligenciaArtificial #Statistics #DataScience #MachineLearning #DeepLearning #BigData #ArtificialIntelligence #Maths #Stats #Econometrics #DataAnalysis #dataviz #Rstatses https://t.co/COWS9ZyWfX  [↪](https://twitter.com/dataandme/status/1189648938560819201)

<!-- -->

> **shannonkcheng** (@shannonkcheng; 0/2): Hey #rstats folks - student I’m working with is looking for a 📦 to sim clinical trials data; I’ve used simstudy & survsim, but she specifically wants to look at drop out rates / outcomes.  Any ideas? I’ve looked at MSToolkit & Mediana but not sure how easy. Cc @rladiestoronto  [↪](https://twitter.com/dataandme/status/1189711979713310721)

<!-- -->

> **Alex Bajaña** (@AlexBajaa5; 0/2): #rstats #rmarkdown #leeds #data #DataScience #DataAnalytics https://t.co/kpzdje28NU  [↪](https://twitter.com/dataandme/status/1189643367405309959)

<!-- -->

> **lisa lendway** (@lisalendway; 0/1): How to prevent fading out of scatter points and preserve text label position in Plotly? #tidyverse #rstats https://t.co/rNne7weBDq  [↪](https://twitter.com/dataandme/status/1189694781867397120)

<!-- -->

> **R Data Berlin** (@rdataberlin; 0/1): Is there a better way to do a group_by for each value in a list? #tidyverse #rstats https://t.co/R00BVaecJJ  [↪](https://twitter.com/dataandme/status/1189689745200295936)

<!-- -->

> **\abs{Z-score} \gt 1.96 Yang** (@zhiiiyang; 0/1): Extracting beta coefficient from group_map() #tidyverse #rstats https://t.co/3FwbtymHuB  [↪](https://twitter.com/dataandme/status/1189649485351268352)

<!-- -->

> **Meridith Bartley** (@AlwaysScientist; 0/1): How to write a variable to avoid dollar sign notation in ggplot aes() #tidyverse #rstats https://t.co/QrUjvSIxnZ  [↪](https://twitter.com/dataandme/status/1189648222437986305)

<!-- -->

> **Dr. GP Pulipaka** (@gp_pulipaka; 0/1): Our EOI for a multi-omics workshop in Perth 3-5 Feb 2020 (3 day including bring our own data) closes Nov 4! Get in quick! https://t.co/ZEklSfHVVG #rstats #microbiome #omics #dataintegration https://t.co/0KKI0OuWWQ  [↪](https://twitter.com/dataandme/status/1189712805198356482)

<!-- -->

> **Mɪʟᴏꜱ Pᴏᴘᴏᴠɪᴄ** (@milos_agathon; 0/1): https://t.co/rwhgMnn909
Añadimos a los históricos de Retro y Chispazo (pronto también Melate) una columna que nos dice cuántos números se repitieron respecto al resultado anterior!
Éxito!
@pronosticos_mx
#suerte #pakin #data #Rstats #analytics #FelizMiercoles #DataScience https://t.co/FfaIf82Bkt  [↪](https://twitter.com/dataandme/status/1189709848679723008)

<!-- -->

> **Niels Chr. Hansen** (@nielschr_; 0/1): Looking for some guides on using xgboost for regression...any decent ones? All I’m finding are classification examples #rstats  [↪](https://twitter.com/dataandme/status/1189685710715944960)

<!-- -->

> **We are R-Ladies** (@WeAreRLadies; 0/1): Any of you #Rstats people deal much with three-way interactions? We have a physiological question that really does feel like a two-way interaction that may have further effect modification based on sex. Sample size is only ~400. Non-sex (and outcome) variables are continuous.  [↪](https://twitter.com/dataandme/status/1189685640121577472)

<!-- -->

> **Aarron Hecksher** (@AAHecksher; 0/1): CRAN updates: colocalization GENEAread tinytex https://t.co/y5W2NTKSXT #rstats  [↪](https://twitter.com/dataandme/status/1189678915108982785)

<!-- -->

> **Alexander Pico** (@xanderpico; 0/1): CRAN updates: generalCorr https://t.co/y5W2NTKSXT #rstats  [↪](https://twitter.com/dataandme/status/1189648708989865985)

<!-- -->

> **Dikayo Data** (@dikayodata; 0/1): #bioconductor #rstats https://t.co/m64An7Luq9  [↪](https://twitter.com/dataandme/status/1189654336286138368)

<!-- -->

> **Big Data Analytics** (@BigData_Fr; 0/1): Ya no hay mañana. Hoy es el 7mo juego de la #WorldSeries. Aqui la Diferencia de Carreras para ambos equipos durante la temporada regular #rstats https://t.co/qhZylFdSK6  [↪](https://twitter.com/dataandme/status/1189673034807943169)

<!-- -->

> **Marck Vaisman** (@wahalulu; 0/1): Need to simulate from a high dim sparse multivariate normal in R. sparseMVN is a good option, but are there any newer techniques? #EconTwitter #rstats #marketingAcads @paulbuerkner @grant_mcdermott @betanalpha @ProfAnirban @shoshievass  [↪](https://twitter.com/dataandme/status/1189671010087424000)

<!-- -->

> **tidyverse tweets** (@tidyversetweets; 0/1): In the fantastic ggimage pkg, is there a way to read in the image files once, store them in variables, and use them as the image aes? All the tutorials I’ve seen use web links or file paths. I’ve read the images in with magick, but can’t see how to use them in the aes #rstats  [↪](https://twitter.com/dataandme/status/1189651680624812034)

<!-- -->

> **Xtophe** (@Xtophe_Bontemps; 0/1): One of my absolute favorite #rstats folks and one of the most fun people in NYC TMTL @D3MPROF has his newest data driven NYU Business school class online.
>
It is a must bookmark
>
https://t.co/iVzSSWft7r  [↪](https://twitter.com/dataandme/status/1189645876022693888)

<!-- -->

> **#rstats data** (@rstatsdata; 0/1): #rstats I'm sure there's an elegant solution that I'm just totally missing. How do I create a unique episode_ID that increases by 1 for instances where episode_flag == "new" but just repeats the value from the row above when episode_flag == "same"?
>
https://t.co/Dl5ZtAiE7J  [↪](https://twitter.com/dataandme/status/1189641234639400961)

<!-- -->

> **CBat** (@ImADataGuy; 0/1): ¿Qué nos muestra el #ITAEE de @INEGI_INFORMA respecto al sector de la construcción?
>
Aquí el gráfico de @CANADEVINAC
>
👇🏼
>
#rstats #rstudio https://t.co/vcisKEqvTl  [↪](https://twitter.com/dataandme/status/1189638191889956865)

<!-- -->

> **Claudio Pacheco** (@claudiodanielpc; 0/1): The interest in NYC squirrels seems to have increased recently #rstats  [↪](https://twitter.com/dataandme/status/1189611721301708800)

<!-- -->

> **Urban Demographics🚡** (@UrbanDemog; 0/0): @rosidi2610 Let's check out this cool #rstats package! 🤘  [↪](https://twitter.com/dataandme/status/1189714462640488449)

<!-- -->

> **Cédric Scherer** (@CedScherer; 0/0): How to merge two datasets together when one set has multiple instances of something I want to merge #tidyverse #rstats https://t.co/56eM2y9tdu  [↪](https://twitter.com/dataandme/status/1189713662472335360)

<!-- -->

repository: batpigandme/twitter-blogdown

repo_name: batpigandme/twitter-blogdown


text:
---
id: 887
title: 'It could be me. (Draw #08)'
date: 2010-04-26 10:16:17 +00:00
author: mark
layout: post
tags:
  - it could be me
  - lottery
---
And, lo, not a penny was won.

Numbers drawn: **1, 4, 12, 23, 33, 40** + bonus ball **35**

My Numbers:  **6, 9, 16, 23, 27, 32**

No. of matching numbers: **1**

Price of ticket: **£1**

Amount won: **£0**

Financial outcome: **-£1**

Which all, rather obviously, means that the two balances look like this…

**Lottery balance: £-8  //  Not A Lottery balance: £8**

license: MIT

branch_name: refs/heads/master

extension: md

gha_language: CSS

gha_license_id: NOASSERTION

license_type: permissive


text:
app.configure([env], callback)

Conditionally invoke `callback` when `env` matches `app.get('env')`,
aka `process.env.NODE_ENV`. This method remains for legacy reasons, and is effectively
an `if` statement as illustrated in the following snippets. These functions are not
required in order to use `app.set()` and other configuration methods.

```js
// all environments
app.configure(function () {
  app.set("title", "My Application");
});

// development only
app.configure("development", function () {
  app.set("db uri", "localhost/dev");
});

// production only
app.configure("production", function () {
  app.set("db uri", "n.n.n.n/prod");
});
```

Is effectively sugar for:

```js
// all environments
app.set("title", "My Application");

// development only
if (app.get("env") === "development") {
  app.set("db uri", "localhost/dev");
}

// production only
if (app.get("env") === "production") {
  app.set("db uri", "n.n.n.n/prod");
}
```

gha_license_id: MIT


text:
---
title: DataVec Operations
short_title: Operations
description: Implementations for advanced transformation.
category: DataVec
weight: 3
---

## Usage

Operations, such as a `Function`, help execute transforms and load data into DataVec. The concept of operations is low-level, meaning that most of the time you will not need to worry about them.

## Loading data into Spark

If you're using Apache Spark, functions will iterate over the dataset and load it into a Spark `RDD` and convert the raw data format into a `Writable`.

```java
import org.datavec.api.writable.Writable;
import org.datavec.api.records.reader.impl.csv.CSVRecordReader;
import org.datavec.spark.transform.misc.StringToWritablesFunction;

SparkConf conf = new SparkConf();
JavaSparkContext sc = new JavaSparkContext(conf)

String customerInfoPath = new ClassPathResource("CustomerInfo.csv").getFile().getPath();
JavaRDD<List<Writable>> customerInfo = sc.textFile(customerInfoPath).map(new StringToWritablesFunction(rr));
```

The above code loads a CSV file into a 2D java RDD. Once your RDD is loaded, you can transform it, perform joins and use reducers to wrangle the data any way you want.

## Available ops

---

### AggregableCheckingOp
 [[source]](https://github.com/eclipse/deeplearning4j/tree/master/datavec/datavec-api/src/main/java/org/datavec/api/transform/ops/AggregableCheckingOp.java)

Created by huitseeker on 5/8/17.

---

### AggregableMultiOp
 [[source]](https://github.com/eclipse/deeplearning4j/tree/master/datavec/datavec-api/src/main/java/org/datavec/api/transform/ops/AggregableMultiOp.java)

It is used to execute many reduction operations in parallel on the same column, datavec#238

Created by huitseeker on 5/8/17.

---

### ByteWritableOp
 [[source]](https://github.com/eclipse/deeplearning4j/tree/master/datavec/datavec-api/src/main/java/org/datavec/api/transform/ops/ByteWritableOp.java)

supports a conversion to Byte.

Created by huitseeker on 5/14/17.

---

### DispatchOp
 [[source]](https://github.com/eclipse/deeplearning4j/tree/master/datavec/datavec-api/src/main/java/org/datavec/api/transform/ops/DispatchOp.java)

Created by huitseeker on 5/14/17.

---

### DispatchWithConditionOp
 [[source]](https://github.com/eclipse/deeplearning4j/tree/master/datavec/datavec-api/src/main/java/org/datavec/api/transform/ops/DispatchWithConditionOp.java)

before dispatching the appropriate column of this element to its operation.

Created by huitseeker on 5/14/17.

---

### DoubleWritableOp
 [[source]](https://github.com/eclipse/deeplearning4j/tree/master/datavec/datavec-api/src/main/java/org/datavec/api/transform/ops/DoubleWritableOp.java)

supports a conversion to Double.

Created by huitseeker on 5/14/17.

---

### FloatWritableOp
 [[source]](https://github.com/eclipse/deeplearning4j/tree/master/datavec/datavec-api/src/main/java/org/datavec/api/transform/ops/FloatWritableOp.java)

supports a conversion to Float.

Created by huitseeker on 5/14/17.

---

### IntWritableOp
 [[source]](https://github.com/eclipse/deeplearning4j/tree/master/datavec/datavec-api/src/main/java/org/datavec/api/transform/ops/IntWritableOp.java)

supports a conversion to Integer.

Created by huitseeker on 5/14/17.

---

### LongWritableOp
 [[source]](https://github.com/eclipse/deeplearning4j/tree/master/datavec/datavec-api/src/main/java/org/datavec/api/transform/ops/LongWritableOp.java)

supports a conversion to Long.

Created by huitseeker on 5/14/17.

---

### StringWritableOp
 [[source]](https://github.com/eclipse/deeplearning4j/tree/master/datavec/datavec-api/src/main/java/org/datavec/api/transform/ops/StringWritableOp.java)

supports a conversion to TextWritable.
Created by huitseeker on 5/14/17.

---

### CalculateSortedRank
 [[source]](https://github.com/eclipse/deeplearning4j/tree/master/datavec/datavec-api/src/main/java/org/datavec/api/transform/rank/CalculateSortedRank.java)

CalculateSortedRank: calculate the rank of each example, after sorting example.
For example, we might have some numerical "score" column, and we want to know for the rank (sort order) for each
example, according to that column.

The rank of each example (after sorting) will be added in a new Long column. Indexing is done from 0; examples will have
values 0 to dataSetSize - 1.

Currently, CalculateSortedRank can only be applied on standard (i.e., non-sequence) data.
Furthermore, the current implementation can only sort on one column

##### transform
```java
public Schema transform(Schema inputSchema)
```

- param newColumnName    Name of the new column (will contain the rank for each example)
- param sortOnColumn     Name of the column to sort on
- param comparator       Comparator used to sort examples

##### outputColumnName
```java
public String outputColumnName()
```

The output column name
after the operation has been applied

- return the output column name

##### columnName
```java
public String columnName()
```

The output column names
This will often be the same as the input

- return the output column names

gha_language: HTML


text:
# SQL Help Scripts

List of quick help scripts:

#### Backup and Restore
1. [Backup File Info](https://github.com/ipranjic22/sql-help-scripts/blob/main/Backup_and_Restore/Backup_file_info.sql) (Where is my .bak file?)

#### Index
2. [Index Rebuild](https://github.com/ipranjic22/sql-help-scripts/blob/main/Index/Index_rebuild_cursor.sql) (Using Cursor)

## License
[MIT](https://choosealicense.com/licenses/mit/)

branch_name: refs/heads/main


text:
---
authors:
- JingsongLi: null
  name: Jingsong Lee
date: "2022-10-13T08:00:00Z"
excerpt: The Apache Flink Community is pleased to announce a bug fix release for Flink
  Table Store 0.2.
title: Apache Flink Table Store 0.2.1 Release Announcement
aliases:
- /news/2022/10/13/release-table-store-0.2.1.html
---

The Apache Flink Community is pleased to announce the first bug fix release of the Flink Table Store 0.2 series.

This release includes 13 bug fixes, vulnerability fixes, and minor improvements for Flink Table Store 0.2.
Below you will find a list of all bugfixes and improvements. For a complete list of all changes see:
[JIRA](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352257).

We highly recommend all users upgrade to Flink Table Store 0.2.1.

# Release Artifacts

## Binaries

You can find the binaries on the updated [Downloads page](/downloads.html).

# Release Notes

        Bug

[FLINK-29098] -         StoreWriteOperator#prepareCommit should let logSinkFunction flush first before fetching offset

[FLINK-29241] -         Can not overwrite from empty input

[FLINK-29273] -         Page not enough Exception in SortBufferMemTable

[FLINK-29278] -         BINARY type is not supported in table store

[FLINK-29295] -         Clear RecordWriter slower to avoid causing frequent compaction conflicts

[FLINK-29367] -         Avoid manifest corruption for incorrect checkpoint recovery

[FLINK-29369] -         Commit delete file failure due to Checkpoint aborted

[FLINK-29385] -         AddColumn in flink table store should check the duplicate field names

[FLINK-29412] -         Connection leak in orc reader

        Improvement

[FLINK-29154] -         Support LookupTableSource for table store

[FLINK-29181] -         log.system can be congiured by dynamic options

[FLINK-29226] -         Throw exception for streaming insert overwrite

[FLINK-29276] -         Flush all memory in SortBufferMemTable.clear

repository: apache/flink-web

branch_name: refs/heads/asf-site

repo_name: apache/flink-web


text:
---
layout: post
title: "Large query string values and Azure"
date: 2014-06-26
categories: .net
---

By default Microsoft IIS accepts a maximum query string length of 2048 characters.  If there is a query string received by the IIS with more than 2048 characters, it will throw a 404.15 - Query String too long error.

If your running a service on Azure and are running into issues sending very large values over http via a get request.  Then you will have to set the maxQueryString property value to compensate for your request.

Unfortunately many answers to this problem fail to address when working on an azure instance.  In that context you will need to upload a Web.config file to the wwwroot folder of your instance. Under the system.webServer, security section, add the requestFiltering section.  Under the requestFiltering section, add the requestLimits tag with your desired maxQueryString value.  Now your web service or web site will be able to accept a maximum query string length you desire.  In the example below I have it set to accept a maxQueryString value of "10000" characters.  Even if you set a big value for maximum query string, there is a limit for each browser which is handling the url and the query string.

{% gist andredublin/802d5c0596e2a3275fbe %}

The best practice is to limit the maxQueryString size as much as possible, to avoid any injection attacks. You can give a value based on your requirements, but keep in mind that alllowing long query strings is a potential security risk and bad design.

## Comments

extension: markdown


text:
---
title: Incremental Loading
page_title: Incremental Loading
description: Check our "Incremental Loading" documentation article for RadListView for UWP control.
slug: radlistview-incremental-loading
tags: radlistview,listview,data,virtualization,incremental,loading
published: True
position:
---

# Incremental Loading

This topic describes how to enable incremental data loading in RadListView.

To enable incremental loading, you have to use as ItemsSource a collection that implements the **ISupportIncrementalLoading** interface. You can also take advantage of the **IncrementalLoadingCollection** which is our default implementation.

The **IncrementalLoadingMode** property specifies how new items are requested. Two loading modes are supported:

- **Automatic**: Scrolling to the end of the control automatically loads more items.
- **Explicit**: The user has to explicitly request more items by click/tap on the button.

The RadListView also supports LoadMoreDataCommand that by default is executed to load new data. You can find more information about the list view commands [here]({% slug radlistview-commands %}).

## Using IncrementalLoading Collection

This example demonstrates how to use the incremental loading functionality. Please note that the approach works for both Automatic and Explicit loading modes.

Here is the RadListView XAML declaration:

	<telerikDataControls:RadListView x:Name="listView" IncrementalLoadingMode="Explicit" ItemsSource="{Binding Items}">
		<telerikDataControls:RadListView.DataContext>
    		<local:ViewModel/>
		</telerikDataControls:RadListView.DataContext>
	    <telerikDataControls:RadListView.ItemTemplate>
	        <DataTemplate>
	            <TextBlock Text="{Binding Text}"/>
	        </DataTemplate>
	    </telerikDataControls:RadListView.ItemTemplate>
	</telerikDataControls:RadListView>

Where:

	xmlns:telerikDataControls="using:Telerik.UI.Xaml.Controls.Data"

This is the view model class:

 	public class ViewModel
    {
        private int currentCount = 0;

        public ViewModel()
        {
            this.Items = new IncrementalLoadingCollection<Item>(this.GetMoreItems) { BatchSize = 5 };
        }

        public IncrementalLoadingCollection<Item> Items { get; set; }

        private async Task<IEnumerable<Item>> GetMoreItems(uint count)
        {
            await Task.Delay(1000);
            var result = Enumerable.Range(this.currentCount, (int)count).Select(x => new Item { Text = "item " + x }).ToList();
            currentCount += (int)count;
            return result;
        }
    }

	public class Item
    {
        public string Text { get; set; }
    }

Here is the result:

![RadListView incremental loading](images/listview-data-virtualization.png "RadListView Incremental Loading")

## Using LoadMoreDataCommand

This example demonstrates how you can utilize LoadMoreDataCommand to load items - the approach works for both Automatic and Explicit loading modes.

Here is the RadListView XAML declaration:

	<telerikDataControls:RadListView x:Name="ListView" ItemsSource="{Binding Items}" />

This is the view model class:

	public class ViewModel
	{
		public ViewModel()
		{
			this.Items = new ObservableCollection<int>();
			this.AddItems(40);
		}

		public ObservableCollection<int> Items { get; }

		public void AddItems(int count)
		{
			for (int i = 0; i < count; i++)
			{
				this.Items.Add(this.Items.Count);
			}
		}
	}

Here is the LoadMoreDataCommand implementation:

	public class CustomLoadOnDemandCommand : LoadMoreDataCommand
	{
		private int lodCounter;

		public CustomLoadOnDemandCommand()
			:base()
		{
			this.Id = CommandId.LoadMoreData;
		}

		public override bool CanExecute(object parameter)
		{
			LoadMoreDataContext context = (LoadMoreDataContext)parameter;
			LoadOnDemandCommand.ViewModel viewModel = (LoadOnDemandCommand.ViewModel)context.DataContext;

			bool canExecute = viewModel.Items.Count < 100;
			return canExecute;
		}

		public async override void Execute(object parameter)
		{
			base.Execute(parameter);

			LoadMoreDataContext context = (LoadMoreDataContext)parameter;
			LoadOnDemandCommand.ViewModel viewModel = (LoadOnDemandCommand.ViewModel)context.DataContext;
			this.lodCounter++;

			if (this.lodCounter % 3 == 0)
			{
				// If we do not need to get new data asynchronously, we can add the new items right away.
				viewModel.AddItems(15);
			}
			else
			{
				// If we need to get new data asynchronously, we must manually update the loading status.

				this.Owner.ChangeDataLoadingStatus(BatchLoadingStatus.ItemsRequested);

				// Mimic getting data from server asynchronously.
				await Task.Delay(2000);

				viewModel.AddItems(15);

				this.Owner.ChangeDataLoadingStatus(BatchLoadingStatus.ItemsLoaded);
			}
		}
	}

Lastly, add the defined above command to the ListView Commands collection:

	this.ListView.Commands.Add(new CustomLoadOnDemandCommand());

	this.DataContext = new ViewModel();

repository: telerik/uwp-docs

repo_name: telerik/uwp-docs


text:
# Flammulaster siparius (Fr.) Watling, 1967 SPECIES

#### Status
ACCEPTED

#### According to
The Catalogue of Life, 3rd January 2011

#### Published in
Notes R. bot. Gdn Edinb. 28(1): 67 (1967)

#### Original name
Agaricus siparius Fr., 1821

### Remarks
null

repository: mdoering/backbone

repo_name: mdoering/backbone


text:
#     Github

Get a release

**Note:** This returns an `upload_url` key corresponding to the endpoint for uploading release assets. This key is a [hypermedia resource](https://developer.github.com/v3/#hypermedia).

Method: Get

OperationID: repos/get-release

EndPoint:

/repos/{owner}/{repo}/releases/{release_id}

repository: Ayehu/custom-activities

path: /Github/repos/GH Get a release/Readme.md

gha_language: C#

repo_name: Ayehu/custom-activities


text:
### Overview

The Metropolitan Atlanta Rapid Transit Authority (or "MARTA") is a public
transportation agency operating heavy rail, bus, and lstreetcar services for citizens
in the metro Atlanta regions of Fulton, Dekalb, and Clayton county. MARTA's daily ridership
peaks at approximately 600,000 passengers with approximately 120,000,000 annual passengers.

MARTA provides a "real-time location" REST API for its bus and rail service. This API provides
the last known system location of every bus and train in transit in the MARTA network.

### API In Detail

MARTA's API is relatively straightforward and simple, which serves as both a strategic strength
and practical weakness. Functionally, MARTA provides two endpoints with similar structure and
purpose - one for rail service, and another for bus service.

Both endpoints return a complete system map with individual stop and station schedule statuses
nested within the output structure. In concrete application, the endpoints function best when
treated as a time-series "snapshot"/"status" of a given system. When called, the API returns
known status and location of trains and buses overlaid on the respective and complete system map.
As a result, determining individual station, stop, and bus/train status is left as an exercise to
the consuming client.

### The Problem

While sufficient for short-term, surface-level insights, the API provided by MARTA suffers from
several key shortcomings that limit its potential impact to the served communities. The key
problems can be grouped roughly along the boundaries of the taxonomy listed below.

* "Ghost trains" -  Often trains on the system map (identified with individual train IDs) will "disappear"
between stations, only returning when during arrival and embarking.

* Data incongruities - Nodes for individual trains and stations list arrival time forecasts as
part of their individual insights, but these insights are often wildly inaccurate and "jumpy".
 A train might be listed as 6 minutes away only to jump up to 20 minutes before returning to
 5 minutes during the next cycle of display updates. This leads to frequent guess work for users
 who are unsure if they've just missed their train or if they're on time.

 * Data absences - Occasionally, the API will simply fail to return any system data for a given station
 or train - it's simply absent in the returned API envelope. When you see a blank screen or "No Data" in
 the station radiators, this is the culprit.

 * Adapted results - As with any system at scale, MARTA's schedules must adapt with circumstances. Currently,
 MARTA's API does *not* adjust API results for daily incidents. Any delays, closures, or even routine maintenace
 schedule adjustments are announced via the official MARTA twitter.

 Examples:

 ![alt text](https://i.imgur.com/xD57K9S.jpg "No data")

 ![alt text](https://i.imgur.com/9KLhrgn.png "Twitter")

 ![alt text](https://i.imgur.com/3yWBJ1v.jpg "Ghost train")

 ![alt text](https://i.imgur.com/9IgWE7G.jpg "Incongruent data")

 ### The Goal

 With these problems in mind, my goal for this project is to consolidate and codify my own habits and workflows as a daily MARTA user
into a single client that can act as a single source of truth for users. I see this as a fun and challenging exercise, but also one of
nominal ethical obligation. MARTA serves as the only means of transportation for many of the poorest and under-served communities in the
region, and MARTA's own capacity to remedy these shortcomings is prone to the political whims of Gold Dome sessions. I would like
this client to serve as the foundational basis for a scalable, insightful portal for individual users to gain reliable insights
into their daily commutes.

From a practical standpoint, that means triangulating the various sources of truth in the MARTA system - static schedules,
API results and Twitter announcements. This data can be paired with individual arrival snapshots to create a rudimentary
model for predictive insights. In plain terms, if we can collect enough arrival/departure insights, we can start picking up
on trends and forecasting results for users.

A practical use-case scenario might be one where a user requests station arrivals
around rush hour, and in addition to the real-time and static arrival data, a warning/notification
is included noting something like _"Red line trains from 5 Point station frequently delayed 5+ minutes at this time."_

repository: trillnye/clj-marta

repo_name: trillnye/clj-marta


text:
# DDNS Slim
> DDNS Client using Node and Typescript

A slim DIY DDNS Client to accompany blog post at
[RickyMendoza.dev - DDNS Client with Node and TypeScript](https://rickymendoza.dev/2020/06/02/ddns-client-using-node-and-typescript/)

## How to build

```sh
npm install
tsc
```

## Usage example

Modify the sites.json to include your sites and endpoint to your DNS Provider. Run with

```sh
npm start
```
- 1.0 Initial Post

## Realeases

## Meta

Ricky Mendoza

Distributed under the MIT license. See ``LICENSE`` for more information.

[https://github.com/rimendoz86/](https://github.com/rimendoz86/)


text:
\mainpage Main Page

---
# Charger 3 click

> Charger 3 Click is a compact add-on board that represents a standalone battery charger with thermal regulation. This board features the TP4056, a complete constant-current/constant-voltage linear charger for single-cell lithium-ion batteries from NanJing Top Power ASIC Corp.

[click Product page](https://www.mikroe.com/charger-3-click)

---

#### Click library

- **Author**        : Stefan Nikolic
- **Date**          : nov 2020.
- **Type**          : I2C type

# Software Support

We provide a library for the Charger3 Click
as well as a demo application (example), developed using MikroElektronika
[compilers](https://www.mikroe.com/necto-studio).
The demo can run on all the main MikroElektronika [development boards](https://www.mikroe.com/development-boards).

Package can be downloaded/installed directly from *NECTO Studio Package Manager*(recommended way), downloaded from our [LibStock™](https://libstock.mikroe.com) or found on [mikroE github account](https://github.com/MikroElektronika/mikrosdk_click_v2/tree/master/clicks).

## Library Description

```
This library contains API for Charger3 Click driver.
```

#### Standard key functions :

- Config Object Initialization function.
```
void charger3_cfg_setup ( charger3_cfg_t *cfg );
```

- Initialization function.
```
CHARGER3_RETVAL charger3_init ( charger3_t *ctx, charger3_cfg_t *cfg );
```

- Click Default Configuration function.
```
void charger3_default_cfg ( charger3_t *ctx );
```

#### Example key functions :

- Charger 3 enable write function.
```
void charger3_enable_write ( charger3_t *ctx );
```

- Charger 3 set current function.
```
uint8_t charger3_set_current ( charger3_t *ctx, float curr_value );
```

- Charger 3 calculate digipot resistance function.
```
float charger3_calc_digipot_res ( charger3_t *ctx );
```

## Examples Description

This example demonstrates the utilization of Charger 3 click.

**The demo application is composed of two sections :**

### Application Init

The application init sets up the UART LOG and I2C communication
drivers. The default configuration disables write protection
and sets the operation mode to charging.

```
void application_init ( void ) {
    log_cfg_t log_cfg;  /**< Logger config object. */
    charger3_cfg_t charger3_cfg;  /**< Click config object. */

    /**
     * Logger initialization.
     * Default baud rate: 115200
     * Default log level: LOG_LEVEL_DEBUG
     * @note If USB_UART_RX and USB_UART_TX
     * are defined as HAL_PIN_NC, you will
     * need to define them manually for log to work.
     * See @b LOG_MAP_USB_UART macro definition for detailed explanation.
     */
    LOG_MAP_USB_UART( log_cfg );
    log_init( &logger, &log_cfg );
    log_info( &logger, " Application Init " );

    // Click initialization.

    charger3_cfg_setup( &charger3_cfg );
    CHARGER3_MAP_MIKROBUS( charger3_cfg, MIKROBUS_1 );
    err_t init_flag = charger3_init( &charger3, &charger3_cfg );
    if ( init_flag == I2C_MASTER_ERROR ) {
        log_error( &logger, " Application Init Error. " );
        log_info( &logger, " Please, run program again... " );

        for ( ; ; );
    }

    Delay_ms( 100 );
    charger3_default_cfg ( &charger3 );
    log_info( &logger, " Application Task " );
    log_printf( &logger, " ------------------------------------\r\n" );
    Delay_ms( 100 );
}
```

### Application Task

Task consists of two operations. First, the desired battery
charging current is set by the user. Since setting the current
doesn't implicitly reveals the resistance, the value
of AD5175 digipot is directly read from the RDAC register,
calculated and displayed on the uart log.

```
void application_task ( void ) {
    float result;

    charger3_set_current( &charger3, 0.4 );
    Delay_ms( 1000 );
    result = charger3_calc_digipot_res( &charger3 );
    log_printf( &logger, " Digipot res value: %.2f ohm\r\n", result );
    log_printf( &logger, " ------------------------------------\r\n" );
    Delay_ms( 5000 );
}
```

## Note

While the resistance of the AD5175 can be directly set and read,
the total resistance value on the PROG pin should be accounted for
( this means an additional 1kohm in series ) setting of the
battery charging current.

The full application code, and ready to use projects can be installed directly from *NECTO Studio Package Manager*(recommended way), downloaded from our [LibStock™](https://libstock.mikroe.com) or found on [mikroE github account](https://github.com/MikroElektronika/mikrosdk_click_v2/tree/master/clicks).

**Other mikroE Libraries used in the example:**

- MikroSDK.Board
- MikroSDK.Log
- Click.Charger3

**Additional notes and informations**

Depending on the development board you are using, you may need
[USB UART click](https://www.mikroe.com/usb-uart-click),
[USB UART 2 Click](https://www.mikroe.com/usb-uart-2-click) or
[RS232 Click](https://www.mikroe.com/rs232-click) to connect to your PC, for
development systems with no UART to USB interface available on the board. UART
terminal is available in all Mikroelektronika
[compilers](https://shop.mikroe.com/compilers).

---

gha_language: C


text:
+++
name= "GitHub Action"
image= "/images/tools/github-action.png"
category= "Tools"
ring= "Strategic"
type="post"
layout="details"
+++

# What is it ?
GitHub Actions makes it easy to automate all your software workflows, now with world-class CI/CD. Build, test, and deploy your code right from GitHub. Make code reviews, branch management, and issue triaging work the way you want.

# Why ?

# For who ?
* [CharlesCD](https://charlescd.io/)
* [Horusec](https://horusec.io/site/)

# Resources
* [homepage](https://github.com/features/actions)

# Resources (Portuguese)
* [Github Actions: uma introdução](https://www.zup.com.br/blog/github-actions-ci-cd)

repository: otaviojava/open-source-tech-radar

repo_name: otaviojava/open-source-tech-radar


text:
---
permalink: /privacy/
title: "Privacy Statement"
layout: splash
---

{{ page.title | markdownify | remove: "
" | remove: "
" }}

Last Updated: 2021-05-31

Websites published by KlearTouch do **not** have any cookies, tracking or analytics. They do **not** collect or store any data.
This website is hosted using GitHub Pages, so [their terms](https://docs.github.com/en/github/site-policy/github-privacy-statement) apply.

Apps published by KlearTouch do **not** send any data to our servers.
They only use your Internet connection to download images, videos or other resources when you request them. Refer to the terms from the owner of these resources.

We welcome feedback via email: [KlearTouch@KlearTouch.com](mailto:KlearTouch@KlearTouch.com)\
We will of course store these emails. Feel free to let us know if you ever want us to delete them.

repository: KlearTouch/website

repo_name: KlearTouch/website


text:
---
title: ".NET Core 3.0 Intrinsics"
published: true
comments: true
header:
  image: /assets/images/intrinsics-header.jpg
title: .NET Core 3.0 Intrinsics in Real Life - (Part 2/3)
date: 2018-08-19 18:26:28 +0300
classes: wide
#categories: coreclr intrinsics
---
As I've described in [part 1]({% post_url 2018-08-18-netcoreapp3.0-intrinsics-in-real-life-pt1 %}) of this series, I've recently overhauled an internal data structure we use at Work® to start using [platform dependent intrinsics](https://github.com/dotnet/designs/blob/master/accepted/platform-intrinsics.md).

If you've not read part 1 yet, I suggest you do so, since we continue right where we left off...

As a reminder, this series is made in 3 parts:

* [The data-structure/operation that we'll optimize and basic usage of intrinsics]({% post_url 2018-08-18-netcoreapp3.0-intrinsics-in-real-life-pt1 %}).
* Using intrinsics more effectively (this post).
* [The C++ version(s) of the corresponding C# code, and what I learned from them]({% post_url 2018-08-20-netcoreapp3.0-intrinsics-in-real-life-pt3 %}).

All of the code (C# & C++) is published under the [bitgoo github repo](https://github.com/damageboy/bitgoo).

### PDEP - Parallel Bit Deposit

We're about to twist our heads with a bit of a challenge: For me, this was a lot of fun, since I got to play with something I knew *nothing* about which turned out to be very useful, and not only for this specific task, but useful in general.

We're going to optimize a subset of this method's performance "spectrum": lower bit counts.
If you go back to the previous iteration of the code, we can clearly see that apart from the one 64 bit `POPCNT` loop up at the top, the ratio between instructions executed and bits processed for low values of `N` doesn't look too good. I summed up the instruction counts from the JIT Dump linked above:

* The 64-bit `POPCNT` loop takes 10 instructions, split into two fragments of the function, processing 64 bits each iteration.
* The rest of the code (31 instructions not including the `ret`!) is spent processing the last <= 64 bits, executing a single time.

While just counting instructions isn't the best profiling metric in the world, it’s still very revealing...
Wouldn't it be great if we could do something to improve that last, long code fragment?
Guess what...
Yes we can! using a weird little instruction called [`PDEP`](https://en.wikipedia.org/wiki/Bit_Manipulation_Instruction_Sets#Parallel_bit_deposit_and_extract) whose description (copy-pasted from [Intel's bible of instructions](https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-instruction-set-reference-manual-325383.pdf) in page 922) goes like this:

> PDEP uses a mask in the second source operand (the third operand) to transfer/scatter contiguous low order bits in the first source operand (the second operand) into the destination (the first operand). PDEP takes the low bits from the first source operand and deposit them in the destination operand at the corresponding bit locations that are set in the second source operand (mask). All other bits (bits not set in mask) in destination are set to zero.

Luckily, it comes with a diagram that makes is more digestible:

![PDEP](../assets/images/pdep.svg)

I know this might be a bit intimidating at first, but what `PDEP` can do for us, in my own words, is this: Process a single 64-bit value (`SRC1`) according to a mask of bits (`SRC2`) and copy ("deposit") the least-significant bits from `SRC1` (or from right to left in the diagram) into a destination register according to the the position of `1` bits in the mask (`SRC2`).
It definitely takes time to wrap your head around how/what can be done with this, and there are many more applications than just this bit-searching. To be honest, right after I read a [paper](http://palms.ee.princeton.edu/PALMSopen/hilewitz06FastBitCompression.pdf) about `PDEP`, which from what I gathered, was the inspiration that led to having these primitives in our processors and an extremely good paper for those willing to dive deeper, I felt like a hammer in search of a nail, in wanting to apply this somewhere, until I remembered I had *this* little thing I need (e.g. this function) and I tried using it, still in C++, about 2 years ago...
It took me a good day of goofing around (I actually started with its sister instruction `PEXT` ) with this on a white-board until I finally saw *a* solution...
*Note*: There might be other solutions, better than what I came up with, and if anyone reading this finds one, I would love to hear about it!

For those of you who don't like spoilers, this might be a good time to grab a piece of paper and try to figure out how `PDEP` could help us in processing the last 64 bits, where we know our target bit is hiding...

If you are ready for the solution, I'll just show the one-liner C# expression that replaces the **31** instructions we saw the JIT emit for us to handle those last < 64 bits in our bitmap all the way town to **13** instructions and just as importantly: with **0** branching:

```csharp
// Where:
// n is the # of the target bit we are searching for
// value is the 64 bits when we know for sure that n is "hiding" within
var offsetOfNthBit = TrailingZeroCount(
                         ParallelBitDeposit(1UL << (n - 1), value);
```

It’s not trivial to see how/why this works just from reading the code, so lets break this down, for an imaginary case of a 16-bit `PDEP` and assorted registers, for simplicity:

As an example, let’s pretend we are are looking for the offset (position) of the 8th `1` bit.
We pass two operands to `ParallelBitDeposit()`:
The `SRC1` operand has the value of `1` left shifted by the bit number we are searching for minus 1, so for our case of `n = 8`, we shift a single `1` bit 7 bits to the left, ending up with:

```ini
0b_0000_0000_1000_0000
```

Our “fake” 16-bit `SRC1` now has a single `1` bit in the **position** that equals our target-bit **count** (This last emphasis is important!)
Remember that by this point in our search function, we have made sure our `n` is within the range `1..64`, so `n-1` can only be `0..63` we we can never shift negative number of bits, or above the size of the register (this can be seen more easily in the full code listing below).

As for `SRC2`,  We load it up with our remaining portion of the bitmap, whose nth lit bit position we are searching for, so with careful mashing of the keyboard, I came up with these random bits:

```ini
0b_0001_0111_0011_0110
```

This is what executing `PDEP` with these two operands does:

![PDEP](../assets/images/pdep-bitsearch-example-animated.svg)

By now, we’ve managed to generate a temporary value where only our original target-bit remains lit, in its original position, so thanks for that, `PDEP`! In a way, we've managed to tweak `PDEP` into a custom masking opcode, capable of masking out the first `n-1` lit bits...
Finally, all that remains is to use the BMI1 `TZCNT` instruction to count the number of `0` bits leading up to our deposited `1` bit marker. That number ends up being the offset of the nth lit bit in the original bitmap! Cool, eh?

Let's look at the final code for this function:

```csharp
using static System.Runtime.Intrinsics.X86.Popcnt;
using static System.Runtime.Intrinsics.X86.Bmi1;
using static System.Runtime.Intrinsics.X86.Bmi2;

public static unsafe int POPCNTAndBMI2(ulong* bits, int numBits, int n)
{
    var p64 = bits;
    int prevN;
    do {
        prevN = n;
        n -= (int) PopCount(*p64);
        p64++;
    } while (n > 0);

    p64--;
    // Here, we know for sure that 1 .. prevN .. 64 (including)
    var pos = (int) TrailingZeroCount(
                        ParallelBitDeposit(1UL << (prevN - 1), *p64));
    return (int) ((p64 - bits) << 6) + pos;
}
```

With the code out of the way, time to see if the whole thing paid off?

Method           |     N |         Mean (ns) | Scaled to "POPCNTAndBMI1"
---------------- |------ |------------------:|-------:
   POPCNTAndBMI2 |     1 |          2.232 |   0.95
   POPCNTAndBMI2 |     4 |          9.497 |   0.62
   POPCNTAndBMI2 |    16 |         40.259 |   0.34
   POPCNTAndBMI2 |    64 |        193.253 |   0.19
   POPCNTAndBMI2 |   256 |      1,581.082 |   0.32
   POPCNTAndBMI2 |  1024 |     23,174.989 |   0.51
   POPCNTAndBMI2 |  4096 |    341,087.341 |   0.82
   POPCNTAndBMI2 | 16384 | 4,979,229.288 |   0.95
   POPCNTAndBMI2 | 65536 | 76,144,935.381 |   0.98

Oh boy did it! results are much better for the lower counts of `N`:

* As expected, the scaling improved with *peak improvement* compared to the previous version at `N==64`, with a 400% speedup compared to the previous version!
* As N grows beyond 64, this version's performance resembles the previous version's more and more (duh!).

All in all, everything looks as we would have expected so far...
Again, for those interested, here's a [gist](https://gist.github.com/9b049a464dc66237500454ed367a79aa) of the JITDump, for your pleasure.

### Loop Unrolling

A common optimization technique we haven't used up to this point, is [loop unrolling/unwinding](https://en.wikipedia.org/wiki/Loop_unrolling):

> The goal of loop unwinding is to increase a program's speed by reducing
> or eliminating instructions that control the loop, such as [pointer arithmetic](https://en.wikipedia.org/wiki/Pointer_arithmetic) and "end of loop" tests on each iteration;[[1\]](https://en.wikipedia.org/wiki/Loop_unrolling#cite_note-1) reducing branch penalties; as well as hiding latencies including the delay in reading data from memory.[[2\]](https://en.wikipedia.org/wiki/Loop_unrolling#cite_note-2) To eliminate this [computational overhead](https://en.wikipedia.org/wiki/Computational_overhead), loops can be re-written as a repeated sequence of similar independent statements.[[3\]](https://en.wikipedia.org/wiki/Loop_unrolling#cite_note-3)

By now, we’re left with only one loop, so clearly the target of loop unrolling is the `POPCNT` loop.
After all, we are potentially going over thousands of bits, and by shoving more `POPCNT` instructions in between the looping instructions, we can theoretically drive the CPU harder.
Not only that, but modern (in this case x86/x64) CPUs are notorious for having internal parallelism that comes in many shapes and forms. For `POPCNT` specifically, we know from [Agner Fog's Instruction Tables](https://www.agner.org/optimize/instruction_tables.pdf) that:

* Intel Skylake can execute certain `POPCNT` instructions on two different execution ports, with a single `POPCNT` latency of 3 cycles, and a reciprocal throughput of 1 cycle, so a latency of `x + 2` cycles as a best case, where `x` is the number of **continuous independent** `POPCNT` instructions.
* AMD Ryzen can execute up to 4 `POPCNT` instructions in 1 cycle, with a latency of 1 cycle, for **continuous independent**  `POPCNT` instructions, which is even more impressive (I've not yet been able to verify this somewhat extravagant claim...).

These numbers were measured on real CPUs, with very specific benchmarks that measure single independent instructions. They should **not** be taken as a target performance for **our** code, since we are attempting to solve a real-life problem, which isn't limited to a single instruction and has at least SOME dependency between the different instructions and branching logic on top of that.
But the numbers do give us at least one thing: motivation to unroll our `POPCNT` loop and try to get more work out of the CPU by issuing independent `POPCNT` on different parts of our bitmap.

Here's the code that does this:

```csharp
using static System.Runtime.Intrinsics.X86.Popcnt;
using static System.Runtime.Intrinsics.X86.Bmi1;
using static System.Runtime.Intrinsics.X86.Bmi2;

public static unsafe int POPCNTAndBMI2Unrolled(ulong* bits, int numBits, int n)
{
    var p64 = bits;
    for (; n >= 256; p64 += 4) {
        n -= (int) (
            PopCount(p64[0]) +
            PopCount(p64[1]) +
            PopCount(p64[2]) +
            PopCount(p64[3]));
    }
    var prevN = n;
    while (n > 0) {
        prevN = n;
        n -= (int) PopCount(*p64);
        p64++;
    }

    p64--;
    var pos = (int) TrailingZeroCount(
                        ParallelBitDeposit(1UL << (prevN - 1), *p64));
    return (int) ((p64 - bits) * 64) + pos;
}
```

We had to change the code flow to account for the unrolled loop, but all in all this is pretty straight forward, so let's see how this performs:

| Method             | N     |         Mean (ns) | Scaled to POPCNTAndBMI2 |
| ------------------ | ----- | ----------------: | -------------------: |
| POPCNTAndBMI2Unrolled | 1     |          2.249 |                 1.04 |
| POPCNTAndBMI2Unrolled | 4     |         10.904 |                 1.15 |
| POPCNTAndBMI2Unrolled | 16    |         50.368 |                 1.11 |
| POPCNTAndBMI2Unrolled | 64    |        208.272 |                 1.13 |
| POPCNTAndBMI2Unrolled | 256   |      1,580.026 |                 0.99 |
| POPCNTAndBMI2Unrolled | 1024  |     21,282.905 |                 0.92 |
| POPCNTAndBMI2Unrolled | 4096  |    255,186.977 |                 0.74 |
| POPCNTAndBMI2Unrolled | 16384 |  3,730,420.068 |                 0.77 |
| POPCNTAndBMI2Unrolled | 65536 | 56,939,817.593 |                 0.76 |

There are a few interesting things going on here:

* For low bit-counts (`N <= 64`) we can see a drop in performance compared to the previous version. That is totally acceptable: We've made the code longer and more branch-y, and all of this was done in order to gain some serious change on the other side of this benchmark (Also, in reality, no one ever complains that your code used to take 193ns, but is now taking 208ns :).
* In other words: The drop is not horrible, And we hope to make up enough for it on higher bit counts.
* And we are making up for it, kind of... We can see a 33%-ish speedup for `N >= 4096`.

For those interested, here's the [JITDump](https://gist.github.com/c73959ad3dfe31e5d65e6bf273f53211) of this version.

In theory, we should be happy, pack our bags, and call it a day! We've done it, we've squeezed every last bit we could hope to.
**Except we really didn't...**
While it might not be clear from these results alone, the loop unrolling hit an unexpected snag: the performance improvement is actually disappointing.
How can I tell? Well, that's simple: **I'm cheating!**
I've already written parallel C++ code as part of this whole effort (to be honest, I wrote the C++ code two years before C# intrinsics were a thing), and I've seen where unrolled `POPCNT` can go, and this is not it.
Not *yet* at least.

From my C++ attempts, I know we should have seen a ~100% speedup in high bit-counts with loop unrolling, but we are seeing much less than that.

To understand why though, and what is really going on here, you'll have to wait for the next post, where we cover some of the C++ code, and possibly learn more about processors than we cared to know...

## Mid-Journey Conclusions

We've taken our not so bad code at the end of the first post and improved upon quite a lot!
I hope you've seen how trying to think outside the box, and finding creative ways to compound various intrinsics provided by the CPU can really pay off in performance, and even simplicity.

With the positive things, we must also not forget there are some negative sides to working with intrinsics, which by now, you might also begin sensing them:

- You'll need to map which CPUs your users are using, and which CPU intrinsics are supported on each model (even within a single architecture, such as Intel/AMD x64 you'll see great variation throughout different models!).
- You'll sometimes need to cryptic implementation-selection code, that uses the provided `.IsHardwareAccelerated` properties (for example detecting `BMI1` only CPUs vs. `BMI1` + `BMI2` ones) to steer the JIT into the "best" implementation, while praying to the powers that be that the JIT will be intelligent enough to elide the un-needed code at generation time, and still inline the resulting code.
- Due to having multiple implementations, architecture specific *testing* becomes a new requirement.
  This might sound basic to a C++ developer, but less so for C#/CLR developers; this would mean that you need to have access to x86 (both 32 and 64 bit) ,arm32,arm64 test agents and run tests on **all of them** to be able to sleep calmly at night.

All of these are considerations to be taken seriously, especially if you work outside of Microsoft (where there are considerably more resources for testing, and greater impact for using intrinsics at the same time), while considering intrinsics.

In the [next and final post]({% post_url 2018-08-20-netcoreapp3.0-intrinsics-in-real-life-pt3 %}), we'll explore the performance bug I uncovered, and how generally C# compares to C++ for this sort of code...

branch_name: refs/heads/dev

gha_language: JavaScript


text:
# Tic Tac Toe
Simple Tic Tac Toe Game that learns to play.

You can play at: https://repl.it/@ericksong/TicTacToe

I have heard of companies giving a Tic Tac Toe game as a project for potential applicants. As an interviewer, I wanted to assess how well it demonstrated my knowledge and see how long it would take me do to a quick and dirty implementation.

## My Requirements:
1. A console app of standard Tic Tac Toe
2. .NET Core
3. Computer had to make smart moves
4. Computer needs to learn from mistakes
5. Computer should play itself to get smart

## I wanted to answer a few questions:
1. Can I do it?
2. Is this a reasonable assignment just in terms of effort required?
3. What does this assess?
4. Learn stuff

### Can I do it?
I guess the answer is yes. I didn't do unit testing (see below). Does it work correctly? Maybe... That is a non-trivial question, lots of cases. I thought it played a decent game. I actually saw it using a strategy that I had never considered. (Trying to trap the player using the side middle squares.) It is easy to poison the cache by throwing games. However, it should eventually recover from this by factoring in lost games.

### Reasonableness
It took me 2.5 hours to complete the code to the initial check in. That seems decent since most of the examples I have seen don't implement any type of learning which was by far the most time-consuming part, probably 2 hours. However, most applicants would probably do something more than a console app which would probably add another couple of hours to make it pretty. Also, doing unit testing would also add a couple of hours. So, this is about an 8-hour project to do 'right'

### Skill Assessment
This did force me to do several things:

1. General coding abilities and style
2. Creating classes and separation of responsibilities
3. Patterns for learning algorithms
4. Basic User Experience understanding
5. Looping and logic structures
6. Storing and looking up data

### Stuff Learned
1. This is a tricky problem with edge cases, non-trivial, a bit humbling
2. I couldn't find a way to just have the machine learn with no knowledge of how to block and win.
3. Having two random players play against each other just seems to produce more randomness. Maybe this is a factor of not playing enough games and needing to have better logic around eliminating bad games. It could also be due to how I had initial implemented the board matching and it could subsequently work.
4. I enjoy solving the hard/interesting parts of problems and not the ones that I have done before.

Perfect, no. Decent, yes.

I would not consider this ideal code, just one way to implement. There was little to no rework of the initial check in.

#### Note on Testing
I did this for fun and was more interested in the logic than the unit testing. I figure adding unit testing would have double the effort because unit testing the learning code would have been a big hassle and I didn't consider that fun. It would be great if someone else who really enjoys writing unit tests wanted to add that. Maybe another learning program could be written to play against this one to test it (them both) thoroughly.

repository: IntelliTect-Samples/TicTacToe

repo_name: IntelliTect-Samples/TicTacToe


text:
SECD Machine
============

This project is an implementation of an [SECD machine](http://en.wikipedia.org/wiki/SECD_machine)
in x86 assembly.  For details, please refer to:

  P. Henderson, "Functional Programming: Application and Implementation",
  Prentice Hall, 1980.

Requirements
------------

  - A POSIX-compatible operating system
  - [nasm](http://www.nasm.us)
  - [GNU make](http://www.gnu.org/software/make/)
  - [GNU m4](http://www.gnu.org/software/m4/)

Directory Structure
-------------------

The project is split up into the following components:

  - secd: The SECD machine implementation.
  - lispkit: The original and extended LispKit Lisp compilers.
  - util: A collection of useful LispKit Lisp functions.
  - example: Some example LispKit Lisp programs.
  - meta: An SECD-machine implementation in (extended) LispKit Lisp.
  - calc: A simple calculator pogram
  - scheme: A Scheme compiler (incomplete -- see scheme/README.md for details).

See the README.md files in each component for further details.

File Types
----------

### Source File (.lso)

These are LispKit Lisp source files, which may represent a standalone program or
a file to be included.  A program file may be distinguished from an include file
in that a program file must contain a single object which has the form:

    (LETREC <name>
      (<name> LAMBDA <params>
        ... function body ...
        )

      ... other definitions ...

      )

The `<params>` may either be `NIL` or may be a list of variable names.  If
`<params>` is non-`NIL`, the parameters will be read by the SECD machine from
stdin.  The result of the function indicated by `<name>` will be printed to
stdout.

### Object Files (.lob)

These are compiled source files that are interpreted by the SECD machine.  To
compile a source file, issue the following command:

	make <program>.lob

To execute `<program>.lob`, issue:

	make run-<program>

Issuing the above commands will rebuild the SECD-machine and the LispKit Lisp
compiler, if necessary.  It is not necessary to manually issue a "make" command
for the SECD machine or compiler prior to building a LispKit Lisp program.

The arguments to the program are read from stdin and the result is written to
stdout.  Some programs may read directly from stdin via the `GET` opcode, rather
than having the SECD machine provide parse the input as arguments to the main
function.  Use the string "))" to separate the end of arguments to the main
function from input read by the program itself.

License
-------

Copyright (c) 2009 Bradley W. Kimmel

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


text:
# Docker for Developers - Code in Action Steps

## Chapter 10: Monitoring Docker using Prometheus, Grafana, and Jaeger
This chapter shows how to set up monitoring and alerting using a variety of cloud-native tools.

### Technical Requirements
- [ ] You must have an AWS account with Administrator privileges
- [ ] You must have a working Kubernetes cluster in the cloud, as set up in Chapter 8
- [ ] You must have Git, the AWS CLI, Helm 3.x, and kubectl installed on your workstation
- [ ] Optional: you may use Spinnaker and Jenkins, as set up in Chapter 9 and Chapter 7, to support continuous deployment

### Setting up a demo application - ShipIt Clicker v7
- [ ] Setup Redis using the Bitnami helm charts locally and in the cloud
- [ ] Install ShipIt Clicker to the k8s local learning environment
- [ ] Install ShipIt Clicker to the k8s cloud EKS cluster
- [ ] Configure Jenkins and Spinnaker to point to this chapter (optional)

### Troubleshooting Kubernetes control plane issues with logs
- [ ] Examine the CloudWatch log streams for the EKS k8s cluster

### Storing logs with CloudWatch Logs
- [ ] Use the `install-fluentbit-daemonset.sh` script to ship kubernetes logs to CloudWatch Logs
- [ ] Use the CloudWatch Logs console to examine the logs from the Kubernetes pods
- [ ] Change the CloudWatch log retention period to a shorter time period

### Storing logs for the long term with AWS S3
- [ ] Clone the CloudWatch2S3 project to your workstation
- [ ] Follow its instructions to set up logs streaming from CloudWatch to S3

### Analyzing logs with CloudWatch Insights and Amazon Athena
- [ ] Use CloudWatch Insights to find the number of ShipIt Clicker games played

### Using the liveness, readiness, and startup probes in Kubernetes
- [ ] Force ShipIt Clicker to fail the readiness check with a test installation
- [ ] Uninstall the test installation and revert the change that forced failure

### Exploring Prometheus through its query and graph web interface
- [ ] Use kubectl to port-forward Prometheus to a local port
- [ ] Explore the Prometheus query interface and issue a `node_load1` query

### Adding Prometheus metrics to an application
- [ ] Examine the metrics code in ShipIt Clicker
- [ ] Examine the deployment helm chart for Prometheus annotations
- [ ] Play the ShipIt Clicker game
- [ ] Query prometheus for a custom metric

### Configuring Prometheus alerts
- [ ] Edit the ConfigMap where Prometheus alerts are defined
- [ ] View the alerts via the port-forwarded Prometheus console

### Sending notifications with the Prometheus Alertmanager
- [ ] Create a new app for slack, and configure a new incoming webhook
- [ ] Edit the Prometheus Alertmanager ConfigMap to add a notification
- [ ] Send a test alert via the Prometheus Alertmanager API
- [ ] Re-deploy the broken ShipIt Clicker application and get an alert
- [ ] Uninstall the broken ShipIt Clicker application

### Gaining access to Grafana
- [ ] Use kubectl to retrieve the Grafana ELB and secrets
- [ ] Log into the Grafana console and explore the dashboards
- [ ] Add some community managed dashboards to Grafana

### Adding a new dashboard with a custom query
- [ ] Add a new dashboard
- [ ] Add two widgets to track game stats: deployments and rate of deployments

### Exploring the Jaeger UI
- [ ] Explore Jaeger using the all-in-one Docker image locally

### Exploring the Jaeger client with ShipIt Clicker
- [ ] Examine the chapter10 files to verify we have the right libraries
- [ ] Examine the configuration of Jaeger inside ShipIt Clicker
- [ ] Look at the ShipIt Clicker games controller to see how tracing is done
- [ ] Use docker-compose to run the ShipIt Clicker app locally
- [ ] Inspect traces for ShipIt Clicker run through docker-compose

### Installing the Jaeger Operator
- [ ] Install the Jaeger Operator to your EKS cluster
- [ ] Edit the Jaeger deployment to get it to watch all namespaces
- [ ] Create a Jaeger Operator instance to configure Jaeger on the cluster
- [ ] Apply annotations to the Jaeger query ingress controller to expose it

repository: PacktPublishing/Docker-for-Developers

repo_name: PacktPublishing/Docker-for-Developers


text:
# yasa_classifiers

This repository contains the notebooks used to generate the pre-trained classifier of [YASA sleep staging module](https://raphaelvallat.com/yasa/build/html/generated/yasa.SleepStaging.html). The main GitHub repository of YASA can be found [here](https://github.com/raphaelvallat/yasa).

For more details on the algorithm and its validation, please refer to the [preprint article](https://www.biorxiv.org/content/10.1101/2<IP_ADDRESS>6165v1.abstract).

The datasets can be found on [sleepdata.org](sleepdata.org). You need to request data access to download the datasets. Specifically, training of the sleep staging classifier is done using the following datasets: CCSHS, CFS, CHAT, HomePAP, MESA, MrOS, SHHS.

If you have questions, please contact Dr. Raphael Vallat (<raphaelvallat@berkeley.edu>).

## Steps

To reproduce all the results in the [preprint article](https://www.biorxiv.org/content/10.1101/2<IP_ADDRESS>6165v1.abstract), you need to run the following scripts/notebooks in order:

0. `00_randomize_train_test.ipynb`: randomize NSRR nights to training / testing set. This assumes that you have previously downloaded the NSRR data on your computer (up to 4 TB).
1. `feature_extraction/01_features_*.ipynb`: calculate the features for all the training nights.
2. `02_create_classifiers.ipynb`: train and export the sleep staging classifier.
3. `predict/03_predict_*.ipynb`: apply the algorithm on testing nights.
4. `04_validation_*.ipynb`: evaluate performance on the testing sets (testing set 1 = NSRR, testing set 2 = DREEM).
5. `05_SHAP_importance.py`: calculate the SHAP features importance on the NSRR training set
6. `06_nsrr_demographics.ipynb`: compare the demographics and health data of the NSRR training / testing set.

In addition, the scripts in the `gridsearch` folder perform parameter searchs with cross-validation to find the best hyper-parameters, class weights and temporal smoothing windows.


text:
# Perforce Discord bot

This is a discord bot for my personal Perforce server.

Right now it only supports fancy highlighting of [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) via the "commit message with scope" format. Maybe at some point I'll extend it to support some of the more generic conventional commit formats, but this works for now.

This will probably get extended to more Unreal Engine-specific use cases in the future.

## Potentional Future Features

- Add command to queue all the files in a changelist
- Add command to get latest changelist from user
- Add association of Perforce username with Discord username
- Support more of the different conventional commit formats ("fix: body"). Maybe breaking change, but not really relevant to me.
- Add detection of C++ changes in a changelist and send message telling people to recompile on the next pull.

### Stretch Goals:
- Trigger an auto format C++ via clang-format after a changelist (there might be a way to do this within perforce itself)

repository: hcorion/perforce-discord-bot

repo_name: hcorion/perforce-discord-bot


text:
---
title: Man pages and explain shell
date: 2022-11-22
tags:
  - linux
  - shell
---

- https://explainshell.com/
- https://tldr.sh/
- https://linux.die.net/man/
- https://www.kernel.org/doc/man-pages/
- https://cht.sh/
- https://github.com/troydm/exp (explain shell cli)
- https://github.com/abhagsain/ai-cli (cli for Open API)

## Example

```bash
❯ exp curl -fsSL
curl(1) ---[ transfer a URL ]

-f --------[ -f, --fail
             (HTTP)  Fail  silently  (no  output at all) on server errors. This is mostly done to better enable
             scripts etc to better deal with failed attempts. In normal cases  when  a  HTTP  server  fails  to
             deliver  a  document,  it  returns an HTML document stating so (which often also describes why and
             more). This flag will prevent curl from outputting that and return error 22.
             This method is not fail-safe and there are occasions where non-successful response codes will slip
             through, especially when authentication is involved (response codes 401 and 407). ]

s ---------[ -s, --silent
             Silent or quiet mode. Don't show progress meter or error messages.  Makes Curl mute. ]

S ---------[ -S, --show-error
             When used with -s it makes curl show an error message if it fails. ]

L ---------[ -L, --location
             (HTTP/HTTPS) If the server reports that the requested page  has  moved  to  a  different  location
             (indicated  with  a Location: header and a 3XX response code), this option will make curl redo the
             request on the new place. If used together with -i, --include or  -I,  --head,  headers  from  all
             requested pages will be shown. When authentication is used, curl only sends its credentials to the
             initial host. If a redirect takes curl to a different host, it won't  be  able  to  intercept  the
             user+password.  See  also  --location-trusted  on  how to change this. You can limit the amount of
             redirects to follow by using the --max-redirs option.
             When curl follows a redirect and the request is not a plain GET (for example POST or PUT), it will
             do  the  following  request  with a GET if the HTTP response was 301, 302, or 303. If the response
             code was any other 3xx code, curl will re-send the following request  using  the  same  unmodified
             method. ]

❯ ai ask "transfer a url, follow redirects, be silent, show errors and fail silently"
...
> Command is `curl -s -L --show-error --fail <URL>`
```

repository: lttr/lukastrumm

repo_name: lttr/lukastrumm


text:
* Name: pic_24F08KA101_projects
* Description:

A set of projects & libraries for a **Microchip**
 PIC 24F08KA101 Micro-controller, written in C (xc16 complier).

Project List
-----------------------------------------

| Number | Description | URL Link |
| --- | --- | --- |
| 1 |  Sixteen segment Alphanumeric LED display library | [URL](projects/16Seg) |
| 2 |  ------- |--------- |


text:
ActiveMQ AMQP Example
=============================

This is an example of using Apache ActiveMQ's amqp protocol with the Apache qpid JMS client.

Notes:

* all instructions assume you are executing from the top level directory of this project
* it is assumed you have Apache Maven installed, and that you are familiar with its usage
* assumes you are using Apache ActiveMQ 5.8.0 or later, or JBoss A-MQ 6.0.0 or later

Note: All of this code will run correctly against either Apache ActiveMQ 5.8.0 or JBoss A-MQ 6.0.0 as both
internally use the same ActiveMQ 5.8.0 code base. The instructions on how to start (command line) the brokers from an
ActiveMQ install will **not** work with JBoss A-MQ (e.g. `bin/amq`) as JBoss A-MQ has ActiveMQ
deployed within an Apache Karaf container to allow for runtime updates to configuration information (versus needing to
restart the broker in the case on Apache ActiveMQ). There are additional steps required to use these provided broker
configuration files and deploy them correctly to JBoss A-MQ with its Fabric based configuration system.
JBoss A-MQ's install contains an `extras` directory with a support version of the Apache ActiveMQ 5.8.0 binary
install.

Starting ActiveMQ
-----------------

Start the broker in a shell:

    shell1> mvn -P broker

Alternatively you can start from an Apache ActiveMQ install:

    shell1> <activemq_home>/bin/activemq console xbean:file:conf/activemq-amqp.xml

Start the message consumer in another shell:

    shell2> mvn -P consumer

Start the message producer in another shell:

    shell3> mvn -P producer

The message producer is coded to send 100 messages. The consumer will log the messages it receives, and will timeout
and exit after 120 seconds of inactivity (no messages received).

repository: kevinearls/activemq-amqp-example

repo_name: kevinearls/activemq-amqp-example


text:
# Notebook to Blog

## Overview

**Notebook to Blog** is a Python CLI that makes it easy to go from a Jupyter Notebook to the following outputs:
- A .txt file containing blog text and placeholders
- Images of plots (saved locally)
- Github Gists

While not completely automated, this will make it much easier to create a blog post (e.g., Medium).

## Instructions

1. Install via `pip install notebook-to-blog`.
2. Store your Github username/password somewhere in your filesystem:

```
GITHUB_USER=<username>
GITHUB_PASSWORD=<password>
```
3. Run via `notebook_to_blog <path to Jupyter notebook> <path to output directory> <path to Github credentials>`

For example, running `notebook_to_blog my_notebook.ipynb output .env` will create the following files:

```
output/
  my_notebok.txt
  images/
    image_00.png
```

In addition, Github Gists will be created for each code block encountered.

**my_notebook.txt** looks like the following:

```
Whatever you wrote in Markdown will appear as regular text.

repository: cgrinaldi/notebook-to-blog

gha_language: Jupyter Notebook

repo_name: cgrinaldi/notebook-to-blog