text:
# Worker

Worker runs a single [*Job*](<https://en.wikipedia.org/wiki/Job_(computing)>) in the background, it can do so immediately or at a scheduled time.

Once registered with [QOR Admin](http://github.com/qor/admin), [Worker](https://github.com/qor/worker) will provide a `Workers` section in the navigation tree, containing pages for listing and managing the following aspects of Workers:

  - All *Jobs*.
  - Running: *Jobs* that are currently running.
  - Scheduled: *Jobs* which have been scheduled to run at a time in the future.
  - Done: finished *Jobs*.
  - Errors: any errors output from any Workers that have been run.

The admin interface for a schedulable *Job* will have an additional `Schedule Time` input, with which administrators can set the scheduled date and time.

[![GoDoc](https://godoc.org/github.com/qor/worker?status.svg)](https://godoc.org/github.com/qor/worker)

## Usage

```go
import "github.com/qor/worker"

func main() {
  // Define Worker
  Worker := worker.New()

  // Arguments used to run a job
  type sendNewsletterArgument struct {
    Subject      string
    Content      string `sql:"size:65532"`
    SendPassword string

    // If job's argument has `worker.Schedule` embedded, it will get run at a scheduled time
    worker.Schedule
  }

  // Register Job
  Worker.RegisterJob(&worker.Job{
    Name: "Send Newsletter", // Registerd Job Name
    Handler: func(argument interface{}, qorJob worker.QorJobInterface) error {
      // `AddLog` add job log
      qorJob.AddLog("Started sending newsletters...")
      qorJob.AddLog(fmt.Sprintf("Argument: %+v", argument.(*sendNewsletterArgument)))

      for i := 1; i <= 100; i++ {
        time.Sleep(100 * time.Millisecond)
        qorJob.AddLog(fmt.Sprintf("Sending newsletter %v...", i))
        // `SetProgress` set job progress percent, from 0 - 100
        qorJob.SetProgress(uint(i))
      }

      qorJob.AddLog("Finished send newsletters")
      return nil
    },
    // Arguments used to run a job
    Resource: Admin.NewResource(&sendNewsletterArgument{}),
  })

  // Add Worker to qor admin, so you could manage jobs in the admin interface
  Admin.AddResource(Worker)
}
```

## Things to note

- If a *Job* is scheduled within 2 minutes of the current time, then it will be run immediately.
- It is possible, via the admin interface, to abort a currently running *Job*: view the *Job*'s data via `Workers > Running` or `Workers > All Jobs` and press the `Abort running Job` button.
- It is possible, via the admin interface, to abort a scheduled *Job*: view the *Job*'s data via `Workers > Scheduled` or `Workers > All Jobs` and press the `Cancel scheduled Job` button.
- It is possible, via the admin interface, to update a scheduled *Job*, including setting a new date and time: view the *Job*'s data via `Workers > Scheduled` or `Workers > All Jobs`, update the `Schedule Time` field's value, and press the `Update scheduled Job` button. Please be aware that scheduling a *Job* to a date/time in the past will see the Job get run immediately.

[Worker Demo:  http://demo.getqor.com/admin/workers](http://demo.getqor.com/admin/workers)

## License

Released under the [MIT License](http://opensource.org/licenses/MIT).

repository: qor/worker

repo_name: qor/worker


text:
---
layout: post
title: Episode 15 - Moisturiser and Measures
date: 2019-05-10 15:00:00.000000000 +00:00
categories: episode
episode_number: '015'
episode_title: Moisturiser and Measures
tags: []
content_warnings: Comedic violence, Emotional abuse, Alcoholism, Substance abuse
voiced: []
acast_url: https://play.acast.com/s/stellarfirma/episode15-moisturiserandmeasures
summary: 'Episode 15

Moisturiser and Measures

PLANET - REVIEW
Citizen Line Manager H. Piltz (ID 63277482) reviews the performance of Citizen Employee T. Geistman (ID 362884) and Bio Facsimile David 7 (ID 95847335).

Line manager recommendation: Restrict Consultants access to moisturiser. Reciprocity via Vent Torment. Alternative foot application. Returned damaged Graph Friend. Bathin is dreamy. Deploy the “Party Ender”. Megatron-Polar Alliance noted. Teamwork Needed.'
formats:
  HTML: https://stellarscripts.tumblr.com/transcripts/15
  Google Doc: https://docs.google.com/document/d/1wOdKcO0ZiNjx4cjV3RSAN9ETzM73qr1z1-SN1qXrrQE/edit?usp=sharing
  PDF: https://drive.google.com/file/d/1QYvE6yP3Hi9w3xw6nD55hnaaQvMW3ZRw/view?usp=sharing
sources:
  stellarscripts: http://stellarscripts.tumblr.com/
official: false
wiki_url: https://stellar-firma.fandom.com/wiki/Episode_15_-_Moisturiser_and_Measures

---

### Transcribed 11/25/19 via [stellarscripts](https://stellarscripts.tumblr.com/) on tumblr

#### ALEX

Hi everyone, Alex here. I'd just like to take a moment to thank some of our patrons: Lucy McCully, Kelly Lux Callahan, Jeffery Sumler, Melissa Verble, Greg Lange, Kelsey P, Mobius, Pettycoated Swashbuckler, Christopher Pajo, ASmith. Thank you all. We really appreciate your support. If you'd like to join them, go to www.patreon.com/rustyquill and take a look at our rewards.

------

##### [show theme]

#### IMOGEN

Rusty Quill presents: Stellar Firma.

##### [theme continues]

#### DAVID

__[frustrated noise]__ Still not here... three! *Three* reviews in a row! __[footsteps approach]__ Three! Just... what, what is he-- what is he doing?!

##### [door swooshes open]

#### TREXEL

_(audibly sick)_ *David...*

#### DAVID

Oh! Oh!

#### TREXEL

*David, help me.*

#### IMOGEN

__[beep]__ Bulbous consultant detected.

#### TREXEL

*David, please*.

#### IMOGEN

__[beep]__ Access granted.

#### DAVID

Why are you so bulbous?!

##### [bwoop of lights turning on]

#### TREXEL

*David, I have been drinking moisturiser for the entirety of the time since I was last here. I haven't let any moisture out of my body in about* [__DAVID:__ Uhh...] *16 hours, David, and it hurts.*

#### DAVID

_(nauseous)_ Oh...

#### TREXEL

__[sobbing]__ *It hurts, David.*

#### DAVID

Umm...

#### TREXEL

*David, you have to squeeze me.*

#### IMOGEN

__[beep]__ Please drain your consultant.

#### DAVID

Uhh...

#### TREXEL

*David, you have to squeeze it out of me.*

#### DAVID

Okay. Okay--

#### TREXEL

*David, grab hold of me.*

#### DAVID

Right, yes.

#### TREXEL

*Grab hold of me, David, and squeeze--*

#### DAVID

Right.

#### IMOGEN

*(crosstalk)* __[beep]__ Line manager arriving.

#### HARTRO

_(crosstalk, cheerfully)_ Oh, hello! I-- *oh.* What's... going on here??

#### DAVID

Oh, um...

#### TREXEL

*Hartro! Hartro, I can explain. I can explain. I was supposed to be on time. I learned. I learned my lesson, I*-- *I promise you, Hartro--*

#### HARTRO

Why do you look so... spherical?

#### TREXEL

*I've drank lots of moisturiser and I think it's caught up with me...*

#### HARTO

_(crosstalk)_ Oh, of course you have... ugh.

#### TREXEL

*David... David, squeeze me...*

#### DAVID

_(crosstalk)_ Right. Okay. Three...

#### HARTRO

Wait, wait, wait. Let me get behind the chair.

#### DAVID

Three... two... one...

##### [David and Trexel strain as they squeeze out the moisturiser. It's gross.]

#### HARTRO

Ohh, it's disgusting!

##### [loud splat as moisturiser is expunged]

#### TREXEL

_(satisfied)_ Oh... that's nice...

 That is *disgusting.*

##### [David whoops as he slips and falls on moisturiser, continues to make traumatised noises]

#### TREXEL

Oh, that-- Oh, he's fallen. Ah... oh, that's wonderful. Oh, I'm so flappy now! Look at all my excess skin.

#### HARTRO

What?! What on earth has made you drink moisturiser, Trexel?!

#### TREXEL

I... picked up a bit of a habit, I have to admit. I was getting some for David-- It's not important.

#### HARTRO

Oh my God, you'd be addicted to addiction if you could!

#### TREXEL

I am! I am!

#### DAVID

_(shakily)_ I'm so gooey again...

#### TREXEL

So gooey.

#### DAVID

It's like I've just been born.

#### TREXEL

Ah, it must be nice for you, then! Remember being born?

#### DAVID

It was very-- yes, recent.

#### TREXEL

Lot's of fun. Lot's of fun.

#### HARTRO

Well, guess what? I'm not here to watch moisturiser coming out of a body. I'm-- We're here to do a review.

#### TREXEL

Yes, I'm sorry. I'm sorry.

#### HARTRO

Well, I need to sit.  I'm sitting here. __[sighs]__ Right. Two things I'm upset about that we need to discuss before I can even begin to review your... planet. __[slippery sounds]__ *Urghh.* Right, one. Last weekend. Ah! You were supposed to be somewhere. [__TREXEL:__ *(softly)* Was I?] Yes, in fact-- oh, I remember! You *killed* several parents and created orphans. [__TREXEL:__ Orphans...] Oh, by dumping lava all over their families. [__TREXEL:__ Yes, yes.] Do you remember now?

#### TREXEL

That is coming back to me, yes.

#### HARTRO

And I thought, oh, wouldn't it be nice to give you a chance to go and entertain the orphans in a way to, you know, make good on the horribleness that you've done.

#### TREXEL

Yes, in-- in the marmalade crèche, if I remember correctly.

#### HARTRO

Yes.

#### TREXEL

I... *did* forget.

#### HARTRO

Yes.

#### TREXEL

I did-- I *did* forget.

#### HARTRO

Yes.

#### TREXEL

I went into a vent--

#### HARTRO

You sure you didn't try to just get away?

#### TREXEL

Well-- well, uh, you know what? Hands up in the air? That's possible. But I did go into a vent, and then I *immediately* forgot why I'd gone in there, and then by the time I'd realised that I didn't know why I was supposed to be in there, I tried to get back and I was lost, and it felt like things were moving? And then it got really hot, and then-- and then really cold--

#### HARTRO

_(crosstalk)_ You're like a very, very sad cat, aren't you?

#### TREXEL

A-- a sad cat?

#### HARTRO

Yeah, you know, they always go into ducts and hide.

#### TREXEL

Yes, Hartro, I am in many ways like a sad cat.

#### HARTRO

Oh, yeah, and you got hot and you got cold. _(gleeful)_ That might have had something to do with *me*.

#### TREXEL

What do you mean?

#### HARTRO

Hahaha. I thought, "Oh, look at my tracker! Oh, look, Trexel's not where he's supposed to be!" And I noticed that you were in a duct. [__TREXEL:__ Yes.] So I thought, I could just turn up the heat, just a *little* bit...

#### TREXEL

Did... did you heat my duct?!

#### HARTRO

Oh, oh, oh, I could just turn-- turn the air conditioning on...

#### TREXEL

Chilled my duct?

#### HARTRO

I did.

#### TREXEL

Right.

#### HARTRO

Was it unpleasant?

#### TREXEL

It was very unpleasant.

#### HARTRO

_(feigning sympathy)_ Ooh... *sorry*.

#### TREXEL

_(crosstalk)_ I don't have much skin on my hands anymore. Although now, I've got so much *excess* skin from all the moisturising, I could probably fold some over. So, well, you know. Comme ci, comme ça.

#### HARTRO

Well... that was very bad. But I feel you've paid your just desserts.

#### TREXEL

Paid in skin!

#### HARTRO

Yes. Ooh... sore. Right. And, oh-- problem number the two. You're late again.

#### TREXEL

Yes, I am late.

#### HARTRO

A-*gain*.

#### TREXEL

But-- but it was *because* I was full of moisturiser, and stuck in a number of narrow hallways--

#### HARTRO

But *is* that my problem? [__TREXEL:__ Uhh--] I mean, *is* that something I need to know? Like,"'Oh, is Trexel *maybe* going to drink moisturiser today, and show up late? Ah!" No, that's not normal problems! [__TREXEL:__ Sorry.] The other Design Teams don't *do* this to me, Trexel!

#### TREXEL

They don't?

#### HARTRO

No! They show up! I review planets, it's a pleasant time! We eat cookies!

#### TREXEL

_(genuine curiosity)_ There's cookies??

#### HARTRO

*Yes!* No cookies for you!

#### DAVID and TREXEL

_(simult.)_ What?

#### HARTRO

Yes.

#### DAVID

Uhh.

#### HARTRO

David, strip your foot.

#### DAVID

_(uncomfortable)_ Oh... hey... the one--

#### TREXEL

But he's wearing-- he's wearing a-- he's wearing a onesie! It just goes all the way around!

#### HARTRO

Just rip it.  Are you wearing socks, David?

#### DAVID

*No!* No! Of course not.

#### TREXEL

_(hushed)_ Hartro, he doesn't like socks. Don't go into the--

#### DAVID

And-- and, so--

#### HARTRO

What, even if I say "socks"?

#### DAVID

Ah!

#### HARTRO

Wh-- socks?

#### DAVID

_(more agitatedly)_ Ah!!!

#### HARTRO

What's wrong with socks?!   [__TREXEL:__ Oh, okay.] _(threatening)_ *And make me believe it.*

#### DAVID

Right, ahh... s-s-sorry, Trexel.

#### TREXEL

Eh, I understand.

##### [David whoops as they put their foot in Trexel's mouth]

#### HARTRO

Don't be *sorry*. __[Trexel makes muffled noise]__ *(whispers)* Or I'll say "socks" again.

##### [Trexel muffled speech]

#### TREXEL

_(muffled])_ *Oh, god-- what is the taste of your foot, David?!*

#### HARTRO

David, I don't believe that you're actually trying to put it in.

#### DAVID

Uh-- oh, sorry.

#### HARTRO

It's just kind of hanging around the lip area.

#### DAVID

_(crosstalk)_ Okay. Um... Okay, okay.

__[David shoves foot in more and begins intermittently making grossed out sounds]__

#### HARTRO

I need it-- it needs to be difficult for him to talk.

#### TREXEL

_(muffled)_ *Oughhghhh*.

#### HARTRO

There. Now, Trexel.

##### [Trexel questioning noise]

#### DAVID

_(disturbed)_ His tongue's so smooth...

#### HARTRO

What did you do.

#### TREXEL

_(muffled)_ *I was late.*

#### HARTRO

You've been late a hundred times.

#### TREXEL

_(muffled)_ A hundred times?

#### HARTRO

An even hundred. That's why today is so special.

#### TREXEL

_(muffled)_ *I'm sorry... for being... late*.

#### HARTRO

He's-- he's sorry for being...?

#### TREXEL

_(more desperate)_ *For being late!!!*

#### DAVID

Oh! Eugh! Eugh, he's trying to say late, I can feel that.

##### [incomprehensible muffled Trexel noises]

#### HARTRO

Can you feel the L?

#### DAVID

Yeah.

#### HARTRO

That would be, like-- like a tongue-y... letter.

#### TREXEL

_(muffled)_ I'll--

#### HARTRO

And-- yeah?

#### TREXEL

_(muffled)_ *I'll never do it again.*

#### HARTRO

Okay, do either--

#### TREXEL

_(muffled)_ *I'll never be late again.*

#### HARTRO

Do you believe him?

#### TREXEL

_(muffled)_ *Please, David, please-- David...*

#### DAVID

He's saying "please" to me, and he's not done that before, so I think he does mean it.

##### [muffled Trexel sobs]

#### HARTRO

All right, you can take it out.

##### [slurping sound of David's foot being removed]

#### TREXEL

Auughhhh.

#### DAVID

Eugh!

#### HARTRO

Now, Trexel, I don't enjoy that. [__DAVID:__ Ooh, it's so smooth. Ooh.] I'm very sorry that I *had* to do that, but you very much did it to yourself.

#### TREXEL

David... David, your foot tastes like week-old hot beef on a radiator.

#### DAVID

Um... thank you?

##### [Trexel coughs weakly]

#### HARTRO

Right, onto the planets.

#### IMOGEN

__[beep]__ Consultant design recommendation file.

#### HARTRO

Serafina Mouchet. Ah, that's labyrinths, and robot _(struggles to pronounce)_ gondol-- golden-- gondil...

#### TREXEL

Oh, ah-- Hartro.

#### DAVID

_(crosstalk)_ Uh-- yes.

#### HARTRO

Yes.

#### TREXEL

Two, three, four...

#### DAVID and TREXEL

♪ Golden robot gondoliers! ♫

#### TREXEL

One for you. And one for me!

#### DAVID and TREXEL

♪ Golden robot gondoliers! ♫

#### HARTRO

Wait, what-- what-- What?

#### TREXEL

Where are they?Who are we?

#### DAVID and TREXEL

♪ Golden robot gondoliers! ♫

#### HARTRO

Okay, what are you doing? What are-- you're singing and dancing.

#### TREXEL

Yes.

#### HARTRO

You made up a song?

#### TREXEL

Yes!

#### HARTRO

For this planet?

#### DAVID

The gold-- golden robot...

#### TREXEL

_(crosstalk)_ For the golden robot--

#### HARTRO

This planet does *not* deserve a song!

##### [beat]

#### TREXEL

Oh.

#### HARTRO

You did something very naughty! *You* put a little bit of IMOGEN into these robot-- Don't sing the song. __[Trexel quietly begins reciting the song]__ Don't you dare.

#### TREXEL

Sorry.

#### HARTRO

So you decided to give them a bit of IMOGEN.

#### TREXEL

Just a little bit of IMOGEN!

#### TREXEL

Just a little soupçon of IMOGEN!

#### HARTRO

*No!* IMOGEN is like... Stellar Firma Ltd.'s head. And you thought you could cut off our nose and give it to a different planet?

#### TREXEL

As you say it back to me, that *does* sound bad. Even a dumbed-down version of IMOGEN is, yes, one of the most powerful entities on the-- on-- on the face of the universe, so...

#### HARTRO

Even-- Yes! Even a speck of IMOGEN is *not* dumbed-down. 'Dumb'-- 'dumb' doesn't go in the same sentence as 'IMOGEN'.

#### TREXEL

Right, right, right.

#### HARTRO

In fact, I bet if she's-- well, she *is* listening, isn't she?

#### TREXEL

She's *always* listening!

#### HARTRO

She's *not* going to be happy about that.

#### TREXEL

IMOGEN? How do you feel about me giving a tiny part of you to a client?

#### IMOGEN

__[error buzz]__ Watch it, buster!

#### TREXEL

Not happy. No.

#### HARTRO

No. No, so.

#### TREXEL

Okay. Well, obviously this would have been caught down the line.

#### HARTRO

*You* should have caught this! In fact, David 7, *you* should have caught this!

#### DAVID

Uh-huh.

#### TREXEL

Yes, David.

#### HARTRO

I-- Why I-- _(warning)_ *Stop it,* Trexel. [__TREXEL:__ Yes, David.] Trexel, *I* do the telling off.

#### TREXEL

Sorry.

#### DAVID

Uh...

#### TREXEL

I'm sorry.

#### HARTRO

David 7?

#### DAVID

Yes. I... was sleepy at... the time? ...Sorry.

##### [beat]

#### HARTRO

__[sighs]__ Right. Don't *ever* let Trexel give out any bit of IMOGEN. Just... *don't* let it happen. I shouldn't need to say this. Anyway, one more thing about that last planet, the Build Team have sent something back. Apparently they didn't need it, it was broken. Anyway, here you go. [__TREXEL:__ Oh.] It's some sort of... I don't know.

#### TREXEL

It's a smashed graph friend.

#### DAVID

Oh!

#### TREXEL

David, this is Henry!

#### DAVID

_(feigning concern)_ Oh-- oh, no. Oh, Henry.

#### TREXEL

Oh, poor Henry, smashed in his prime.

#### DAVID

Ohh...

#### TREXEL

I-- I commiserate with you, David. __[compassionately sighs]__ I know--

#### DAVID

_(not sad at all)_ So sad.

#### TREXEL

--that a-- that a person and their graph friend, it's-- it's a bond that... that only rivals the love of a parent. I mean, obviously, *I* have Percy at home, and if anything happened to Percy like this, well, I don't know what I'd do. I'd probably just tear...

#### HARTRO

It's a *toy*.

#### TREXEL

It's a friend! It's a graph friend! You wouldn't understand, Hartro, you had other friends as a child, but me and--

#### HARTRO

_(dryly)_ I have *real* friends.

#### TREXEL

Me and David here... we're different. We're friendless in this world.

#### HARTRO

I'm *definitely* different.

#### TREXEL

I haven't seen Percy in a long time...

#### DAVID

_(quickly)_ Oh, I'm sure he'll turn up. Um, so...

#### HARTRO

Next planet.

#### IMOGEN

__[beep]__ Consultant design recommendation file.

#### HARTRO

Bathin, the Great Duke of Galactonium. [__TREXEL:__ Hmm...] And he is *great*, isn't he? Oh... I love working with him!

#### TREXEL

_(grumbling)_ Yes, yes, yes. What about Bathin? What did he-- what did he think of his planet?

#### HARTRO

Oh, what about him? Oh yes! Oh, um, well, he's filled out a feedback form!

#### DAVID

Oh, that's very helpful!

#### HARTRO

So conscientious of him, isn't it? [__DAVID:__ Mm.] And-- well, I've got it here, and--

#### IMOGEN

__[beep]__

#### HARTRO

Oh. Oh, no, it's just... oh, it's just a photo of him... with h-his autograph. __[Trexel sighs deeply]__ [__DAVID:__ Oh!] That's strange... Lovely, isn't it?

#### DAVID

It's very... Yeah-- What handsome nipples he has!

#### HARTRO

He does! And he's *strong*, isn't he? [__DAVID:__ Mm!] But it looks like he's sensitive as well?

#### DAVID

Yes, big soft arms. _(appreciative)_ Mmm.

#### HARTRO

_(crosstalk)_ Like I feel like you could talk to him if you had a problem.

#### DAVID

Maybe just lay your head on his... chest.

#### TREXEL

Is-is-is there any actual-- any actual *feedback* or is this just... pictures.

#### HARTRO

Oh, uh-- Oh, I-- I took some notes, so I'll just see the...

#### IMOGEN

__[beep]__

#### HARTRO

Oh, no. No, it's just more... more photos of Bathin-- [__TREXEL:__ Oh, I see.] Oh, David 7, look at this.

#### DAVID

Is he-- is he making six cocktails at once?

#### HARTRO

How-- how could you even *do* that?

#### DAVID

I know!

#### HARTRO

__[soft gasp]__ And he makes it look-- he looks like he'd be-- it's *fun*. You know? Like, I want to go over there and... and have one of them.

#### DAVID

He's not even trying.

#### HARTRO

No.

#### DAVID

You know, it's just-- he's talking to you and-- and listening actively, at the same time as making six cocktails at once and then--

#### HARTRO

Trexel! Look!

#### TREXEL

_(alfheartedly)_ Isn't... It's nice.

#### IMOGEN

__[beep]__

#### HARTRO

And-- Oh, and look at this one! Oh, he's studying! [__DAVID:__ Mmm!]  What's he reading there?

#### TREXEL

_(snappish)_ Well, *that* doesn't seem like a real book!

#### HARTRO

But it is! Look, it's right there.

#### DAVID

Well, no it's-- It's quite big. Very small text, actually, I think there's a... So it's a dense read. [__TREXEL:__ *(still slightly bitter)* Well, okay.] But he seems to be really into it, actually...

##### [Hartro sighs dreamily]

##### [beat]

#### TREXEL

So--

#### HARTRO

Trexel, why do you look so angry?

#### TREXEL

Well, I just-- _(whispered fervently)_ I just feel like we're spending a lot of time talking about how good Bathin is, when everybody in this room knows *full well* that I do not get on with Bathin.

#### HARTRO

Oh, you know Bathin?

#### TREXEL

Yes! Yes.

#### HARTRO

You know Bathin?!

#### TREXEL

I do-- You *know* I know Bathin!

#### HARTRO

Oh-- tell me about Bathin!

#### TREXEL

He's-- an-- an old school-- I'm gonna say *friend* of mine, but we didn't get on.

#### HARTRO

__[offended gasp]__ Take it back.

#### TREXEL

I'm sorry, that's how I feel, Hartro.

#### HARTRO

Do you know what? He's a respectable man. He does good things. He has several charities. Do you have a charity? Do you have *one* charity? [__TREXEL:__ I don--] He has seven charities that I can think off the top of my head!

#### TREXEL

What-- what for? What are these charities?

#### HARTRO

Oh-- one charity for... people who don't know where their spaceship is.

#### TREXEL

Well, I suppose--

#### HARTRO

Do you ever have it when you lose your spaceship.

#### TREXEL

Yes. Yes.

#### HARTRO

Well, he's got something for that!

#### TREXEL

Okay.

#### HARTRO

People who don't like to shop.

#### TREXEL

I-I know people hate that.

#### HARTRO

Just the other day, I didn't feel like shopping. I just called up his charity and somebody came and shopped for me. He's a lovely man. He makes the world better, Trexel.

#### TREXEL

Fine, I take it back... Bathin is a wonderful person and a valued client.

#### HARTRO

Hmm. Fair. Was that so hard?

##### [David and Hartro make shocked and disgusted noises]

#### IMOGEN

__[beep]__ Warning: nipples detected. Security alerted.

#### TREXEL

_(crosstalk)_ Yeah! See?

#### HARTRO

No, stop!

#### TREXEL

We've all got handsome nipples!

#### DAVID

Oh, so *dry...*

#### TREXEL

Yes. Well, since you squeezed all the moisture out of me, David, it's dry and flappy now.

#### HARTRO

_(disgusted)_ Put it down. Put it down.

#### TREXEL

Okay. __[Hartro sighs in relief]__ I'll-- I'll hush these puppies away. So, see? Both Bathin and I have very handsome nipples, so it's not *all* his game!

#### DAVID

_(supremely upset)_ Like tiny blue calderas.

#### HARTRO

Oh. Ergh, okay. Let's just shake that off. Uh... back to business!

#### IMOGEN

__[beep]__ Consultant design recommendation file.

#### HARTRO

Psilax Bromius, the party planet! Yes, yes, yes, yes. The ultimate rave in space.

#### TREXEL

Yes! The space rave.

#### HARTRO

Well, I don't know *how,* but you've actually managed to make a party that teenagers *like*.

#### TREXEL

A success! A success, David!

#### DAVID

Oh, we did well!

#### HARTRO

No, no no no, no--

#### TREXEL

We've made a party planet and people love it! Grab the-- champagne! Champagne!

#### HARTRO

No, no, no, no! It's bad! It's terrible! Because all of the teenagers in the galaxy are going to this planet!

#### TREXEL

Well, that's-- that's what we *wanted!* That's what we wanted, a super-- a super cool party with music _(sings)_ ♪ I'm scared of... do dum da dum dum.Dance now. ♫

#### HARTRO

Trexel. *All* of the teenagers are going. We're losing our youth. In fact, some of these teenagers have parents who are *senior executives*!

#### TREXEL

Oh, *no*.

#### HARTRO

It is not good.

#### TREXEL

Not good at all, no! No, we have to stop this! If-- if the senior executives get wind that all of their teens have been enticed onto my rave planet, then-- then-- then I'll never-- I'll never get another decent assignment again! We have to stop this, Hartro, we have to stop this!

#### HARTRO

You don't think I know this?

#### DAVID

We could just end the party! [__TREXEL:__ Yes!] We've got the-- Just end the party. We built that in!

#### TREXEL

We built in a party ender!

#### HARTRO

_(crosstalk)_ If it was that simple, we would do that!

#### TREXEL

But you just-- you just press the party ending button, [__DAVID:__ The moon crashes into the--] and then a moon just *crashes* in and it turns into a *burning star*!

#### HARTRO

...Okay, do you see the problem there?

#### TREXEL

_(muttering to himself)_ Okay, hang on there.

#### HARTRO

I-I-I'd like to just... Yes.

#### TREXEL

_(muttering to himself)_ Well, you crash... crash the moon into the-- and then burning star, [__HARTRO:__ Yeah.] and then everybody dies... _(realisation)_ every-- everybody dies.

#### DAVID

Ahhh.

#### HARTRO

Everybody, [__TREXEL:__ Everybody dies.] including very important teenagers to senior executives.

#### TREXEL

V.I.P. teens.

#### HARTRO

Yes.

#### TREXEL

Right.

#### HARTRO

So we *cannot* do that.

#### TREXEL

Righty-o. Well, in my experience, parties always end about 15 minutes after I arrive, so... __[Hartro gasps]__ so surely it-- it can't be long for this world!

#### HARTRO

That is brilliant.

#### TREXEL

What's brilliant? Have I had a good idea?

#### HARTRO

Oh my gosh, Trexel, I don't know how you've *accidentally* had so many brilliant ideas, but, you go! *You* go! Yes, oh, lovely, yes, oh, I love that.

#### TREXEL

What-- what do you mean? If I go, surely people will-- will think it's even hipper. _(imitates 'cool teen' voice)_ "Trexel's here! Look at this. Look at him. Look at that."

#### HARTRO

Trexel, I'm counting on the fact that that *won't* happen.

#### TREXEL

All right. I'll-- I'll make you a bet.

#### HARTRO

All right!

#### TREXEL

I'll go there and I'll turn up. And yes, parties do tend to end pretty quickly after I finish, but only because they're all, like, "Whoa, this party's *too* hot!"

#### HARTRO

I have an idea.

#### TREXEL

What?

#### HARTRO

Yes. Yes. You'll go this weekend.

#### TREXEL

Yes.

#### HARTRO

Lovely!  [__TREXEL:__ Well-- wh-- wh-- sh--] Fantastic. David 7, I'm brilliant, don't you think?

#### DAVID

Oh, yes, absolutely.

#### HARTRO

'Cause it was horrible, wasn't it?

#### DAVID

The nipples.

#### HARTRO

I wanted to be away.

#### DAVID

_(haunted)_ The nipples.

#### TREXEL

_(offended)_ All-- all right, all right, we can all agree that my nipples maybe aren't the best, but they've got-- they've got something to offer to the world.

#### DAVID

Well, just offer them to the party.

#### TREXEL

Well, okay, yes. We'll-- we'll-- we'll take them to the masses! I'll lift up my shirt to the party and shout, "Hey teens: here's the nips!" and then we'll see what happens. Okay?

#### HARTRO

Ah, I think that's good as done!

#### DAVID

_(softly)_ Yes, settled that one pretty well.

#### TREXEL

Good. Well, I didn't have any plans this weekend so I look forward to it!

#### HARTRO

Yeah. And we know you like to party.

#### TREXEL

I love to party!

#### IMOGEN

Consultant design recommendation file.

#### HARTRO

Lorek Brynison's planet, uh, the armoured polar bears, yes. Oh come-- __[scoffs]__ that should have been so easy? Right?

#### TREXEL

Hartro, I went into this planet *so* carefully. I have *personally* caused two world deaths for these people. [__HARTRO:__ Yes.] So I made it very, very clear to David and to myself when we were designing it, that we can on no account let them know that we are even *thinking* about heating up *any* sort of ice on their planet. And I think I nailed it.

#### HARTRO

You *hid* heaters in accountants.

#### TREXEL

Who looks at accountants? Who looks at accountants?

#### HARTRO

What-- Apparently polar bears! I mean there's flesh, there's blood. I mean, that just says "tasty morsel"!

#### TREXEL

But-- but they were seals! And we made them accountants to make them less enticing!

#### HARTRO

It's still flesh, come on!

#### TREXEL

That is true.

#### DAVID

Uh, yes. Yes. Slurry.

#### TREXEL

Yes. I just *drink*.

#### HARTRO

Moisturiser...

#### TREXEL

Moisturiser, mostly. Very fattening.

#### HARTRO

Hmm.

##### [beat]

#### TREXEL

So they've eaten the accountants.

#### HARTRO

They're very upset.   [__TREXEL:__ Yes. So--] Right? "That old Trexel Geistman, third planet is the charm!"

#### TREXEL

Did they-- did they know it was *me* specifically?

#### HARTRO

Oh yes, they know it was you.

#### TREXEL

Ah.

#### HARTRO

Yeah. *Not* good. Oh, and guess who's reached out to them?

#### TREXEL

Uh-- is it you? Is it you with an apology on my part?

#### HARTRO

Well, of course I've reached out to them. I've smoothed it over like I always do, as best I can. But no, they don't want to see me, because now they're in alliance with the Megatrons! [__TREXEL:__ Oh...] Ha-- the other-- the *other* planets that hate us! This is not good.

#### TREXEL

Yes. To be fair, I'm not at fault for the Megatrons, that was humanity.

#### HARTRO

Well, yes, but we don't want them working together, and they are.

#### TREXEL

No. And they are.

#### HARTRO

And you are at fault for that.

#### TREXEL

And that is--

#### HARTRO

So it's twice as bad.

#### TREXEL

Okie-dokie.

#### HARTRO

I don't understand, David 7.

#### DAVID

Uh, yes?

#### HARTRO

That was an *obvious* warlike situation which Trexel created. Why did--

#### TREXEL

David, are you trying to *sabotage* me?

#### DAVID

_(flatly)_ No.

#### TREXEL

*Et tu*, David!? [__DAVID:__ No.] A viper in my bosom! I clutched you to my breast--

#### DAVID

_(crosstalk)_ Oh. No...

#### HARTRO

_(crosstalk)_ Oh, hold your horses.

#### TREXEL

Like a kindly snake, and it turned out you would turn and *bite me!*

#### DAVID

Just... very sleepy... Sorry! Um...

#### HARTRO

Oh, *and*... and, you requested a particle accelerator.

#### DAVID

Uh...

 [__TREXEL:__ Subservience.] Yet right after the subservience, I think that's when they add the "don't start wars" [__TREXEL:__ Don't-- don't-- don't cause wars. Yep.] and... "don't request particle accelerators". [__TREXEL:__ Yeah, don't-- don't--] It's very clear.

#### TREXEL

Do not mess with the fabric of reality.

#### HARTRO

And yet that-- I have an alert here that's been issued from IMOGEN, that you did do that.

#### TREXEL

_(softly)_ Yes.

#### DAVID

Well... Trexel's very forceful... and __[stammers]__ I also need to support... the designer in their-- their desires, so.

#### HARTRO

You do! You need to work together.  There's not enough teamwork going on! Ooh, that gives me an idea. Ah, yes, I know exactly what I'll do.

#### TREXEL

Right, what-- what are you going to do?

#### HARTRO

Uh, brilliant! Great! Oh, you'll see. Ah... bad job, and... planets bad, and... _(quickly)_ see you later. Bye!

##### [slippery sound as she stands, door swooshes closed]

#### TREXEL

Oh. Bye, Hartro. Bye!

#### DAVID

Bye, Har-- Okay. Um... Oh, I hope that doesn't involve feet.

#### TREXEL

_(solemnly)_ David, it's almost certainly gonna involve feet in some way. [__DAVID:__ Oh, no.] I don't know why and I don't know how, but when Hartro Piltz has an idea and it's to do with me, it's gonna end up with a foot in my mouth.

#### DAVID

Right.

#### TREXEL

But she has a point, David!

#### DAVID

What?

#### TREXEL

Previously, most clones have, you know, given me a bit of a heads up about this sort of stuff. Sure, I've ignored them. Sure, they've been recycled, but they yammer on, saying, "No, Trexel, no, this is against my directives!" _(suspiciously)_ And that hasn't happened a *single* time with you.

#### DAVID

Up... graded models...

#### TREXEL

I know what's going on, David!

#### DAVID

Uh...

#### TREXEL

I know exactly what's happening here! [__DAVID:__ Wh--] You try and trick me and hide it from me, David, but I've got to the core of the idea. [__DAVID:__ Wha--?] I know what this has all been about! [__DAVID:__ What--] Weeks on weeks you've been here and now I understand it! *I need another pint of moisturiser!* That's it, isn't it?! I've got distracted, what are we doing? I'm thirsty. Bye bye!

##### [slippery sound as he stands, door swooshes closed]

#### DAVID

Ahh... um. __[nervous exhale]__ How do I fix this?

##### [show theme starts]

------

#### IMOGEN

Stellar Firma is a podcast distributed by Rusty Quill Ltd. and licenced under a Creative Commons Attribution Noncommercial Sharealike 4.0 International Licence. It was created by Tim Meredith and Ben Meredith, and produced by Lowri Ann Davies with executive producer Alexander J. Newall. In today's episode, I.M.O.G.E.N. was played by Imogen Harris, David 7 was played by Ben Meredith, Hartro Piltz was played by Jenny Haufek, and Trexel Geistman was played by Tim Meredith. The episode was edited by Edward von Aderkas and Alexander J. Newall, with music by Samuel D.F. Jones and artwork by Anika Khan. To subscribe, buy merchandise, or join our Discord, visit rustyquill.com. Rate and review us online, tweet us @TheRustyQuill, join our Reddit community on r/RustyQuill, visit us on Facebook, or email us via<EMAIL_ADDRESS>May the Board preserve and keep you.

license: MIT

branch_name: refs/heads/master

extension: md

gha_language: JavaScript

gha_license_id: MIT

license_type: permissive


text:
[![CircleCI](https://circleci.com/gh/dof-dss/nidirect-site-modules.svg?style=svg)](https://circleci.com/gh/dof-dss/nidirect-site-modules)

# NI Direct custom modules

This repository houses the custom modules required for the NI Direct website.

## Breadcrumbs
- Custom navigation breadcrumbs.

## Cold Weather Payments
- Weather station entity.
- Cold Weather Period field, widget and formatter.
- Search form.
- Search block and token.
- Cold Weather Payments REST API.

## Common
- Cron job for audit date updates.
- Save top_level_theme to node on save.
- Invalidate taxonomy cache tags on node sav or deletion.
- Add additional guidance on LinkIt to WYSIWYG editor.
- Display singular/plural header on Recipes search View.
- Display singular/plural header on Publications search View.
- Provide banner field for application, article, publication and
  health_condition entities.
- Description added to theme/subtheme field for landing pages.
- Switch between entity embed view modes for edit and view conditions.
- Site search block.
- D8NID-347 : Various tweaks to publication search page.
- D8NID-373 : Add entries for site footer links.
- D8NID-426 : Redirect theme taxonomy term urls to landing pages.
- D8NID-460 : Warn user if landing page for subtheme already exists.
- D8NID-471 : Hide top level theme field from node form.
- D8NID-478 : Entity browser - hide authored by, authored on, URL alias, and
  revision log fields.
- D8NID-480 : Adjusted the View and CSS to tweak the Entity browser display
  to prevent overflows.
- D8NID-479 : Entity browser - disable select button until entity is selected.
- D8NID-490 : CKEditor: Improve UX for adding a map (location).
- D8NID-571 : Add headless JS tests to CircleCI to guard against known config
  import issues.
- D8NID-617 : Flags, checkboxes, help text changes.
- D8NID-601 : Linkit: remove option to open link in a new window.
- D8NID-220 : Featured content blocks for homepage.
- D8NID-556 : Allow editors to select from a list of predefined title values
  for telephone plus entries.
- D8NID-635 : JS to rewrite links as spans if current url matches link href.

## Contacts
- AtoZ block.
- Listing controller.
- Tweaks to exposed View form.
- Update View Query to alter query conditions.
- Process View display to alter sort links.

## Custom Blocks
- Article teasers by Topic block.

## Driving Instructor
- Removes sort/order by options from exposed form.
- Display singular/plural header on search View.
- Disable Driving Instructor title field and generate title on node save.
- Prevent selection of parent category terms on node edit form.
- Remove second portion of the postcode if present on search query.
- Hide empty text, result count and sort options on initial display of search
  View.
- Provide custom sort links based on url parameters.

## Error Pages
- Provides template suggestions for error pages.

## GP
- GP entity.
- GP entity forms.
- GP entity and access controllers.
- Lookup lead and member GPs for practices.
- GPs field for practices.
- Disable Practice title field and generate title on node save.
- Validation on Practice form for practice pr surgery name.
- Provides default and hides title field for appointment and prescription
  links on GP Practice form.
- Adds result count and form tweaks to View display.

## Health Conditions
- Blocks for AtoZ and Related Conditions.
- Health Conditions listing controller.
- Update exposed View form with reset link.
- Add summary and related conditions to node view.
- Update node title if viewing alternative health condition.

## Hospital Waiting Times
- Fetch waiting times from API.
- Setting form for configuring API calls.
- Ultimate Cron job for calling API.
- Token for displaying waiting times.
- Theme hook and twig template for displaying waiting times.

## Money Advice Articles
- Migration API configuration for importing Money Advice Service articles.
- Ultimate Cron job calling Migration API.
- Migration process function to alter article fields.

## News
- News listings controller.
- Change date published label.

## Related Content
- Blocks for Recipes by Course, ingredient, season, special diet.
- Display related content on View.
- Set View title to parent theme title.

## Search
- Query adjustments for local dev:
  - Lando works with LANGUAGE_UNDEFINED spellcheck but not with EN. Could be
    env specific so restricted to that environment.

> NB: Review of this recommended once hosting environment is available.

## School Closures
- School closures service and interface.
- C2KSchools closure service.
- School closures settings form.
- Theme hook and twig template.
- School closures Token.
- JS for filtering closure results.

## Webforms
- Webform handler for Quiz Results.
- Theme hooks and twig templates for Quiz Results.

repository: dof-dss/nidirect-site-modules

branch_name: refs/heads/development

gha_language: PHP

repo_name: dof-dss/nidirect-site-modules


text:
11.4: Rating Select on enter example
====================================
**Primary Actor**: User

**Scope**: Ngx-bootstrap DEMO / BS version 3&4

**Goal**: Show user rating with key navigation (with arrows and enter)

Main success scenario:
----------------------
1. User opens Rating demo page
2. User clicks on Select on enter sub-menu
3. User see rating card with info "Rating: N;  Readonly is: false;" and clickable button "Reset rating and status"
4. When user change the mark, then rating changed with appropriate info in the card
5. When user focus on the rating, then after press "right arrow" on the keyboard, rating value increased at 1
6. When user press "left arrow" on the keyboard, rating value decreased at 1
7. When user press "Enter" on the keyboard, then rating saved this state and card info updated with "Rating: N;  Readonly is: true;"
8. When user clicks on "Reset rating and status", then all stars in rating become unselected and card text become "Rating: 0;  Readonly is: false;"
9. When user change the mark, then rating changed with appropriate info in the card

Variations:
-----------
2*. User scroll to Select on enter sub-menu

repository: valor-software/ngx-bootstrap

gha_language: TypeScript

repo_name: valor-software/ngx-bootstrap


text:
LDAP Postfix and Dovecot Auth on Ubuntu 20.04

## Table of Contents

- [Table of Contents](#table-of-contents)
- [About The Project](#about-the-project)
  - [Built With](#built-with)
- [Getting Started](#getting-started)
  - [LDAP](#ldap)
    - [LDAP Installation](#ldap-installation)
    - [Verify the LDAP Search](#verify-the-ldap-search)
    - [Custom Directory Creation and Directory Modification](#custom-directory-creation-and-directory-modification)
    - [Load an Additional Schemas](#load-an-additional-schemas)
    - [Adding Users by LDIF File](#adding-users-by-ldif-file)
    - [LDAP Configuration for Test](#ldap-configuration-for-test)
  - [Postfix](#postfix)
    - [Postfix Installation](#postfix-installation)
    - [Postfix Configuration](#postfix-configuration)
    - [Starting Postfix](#starting-postfix)
    - [SMTP Connection Test](#smtp-connection-test)
    - [LDAP Lookup Test](#ldap-lookup-test)
  - [Dovecot](#dovecot)
    - [Dovecot Installation](#dovecot-installation)
    - [Dovecot Configuration](#dovecot-configuration)
    - [Starting Dovecot](#starting-dovecot)
    - [IMAP Connection & Authentication Test](#imap-connection--authentication-test)
    - [Postfix Integration](#postfix-integration)
    - [Dovecot Deliver Test](#dovecot-deliver-test)
    - [Log Viewing](#log-viewing)

## About The Project

This project made for Authentication of `Lightweight Directory Access Protocol (LDAP)`, `Postfix` and `Dovecot`. **Basically installation of a mailserver**, in order for them to be connected to each other, and help with address lookup and aliases.

* `Ubuntu 20.02`

* `Users don’t have system accounts on the Unix machine`

* `User accounts are virtual accounts stored in an LDAP`

### Built With

* [OpenLDAP](https://www.openldap.org/)
* [Postfix](http://www.postfix.org/)
* [Dovecot](https://www.dovecot.org/)

## Getting Started

### LDAP

This is the directory service that stores the (virtual) user accounts for the mail server.
The LDAP directory needs to keep several information needed by Postfix and Dovecot to work.

- Username & Password for authentication
- __UID__ and __GID__ for managing permissions of the user's __maildir__
- Location of the __maildir__
- List of mail aliases for a given user to allow one user to have several mail adresses.

#### LDAP Installation

```sh
apt-get install slapd ldap-utils
```
After installation, you can see running LDAP port on 389:
```sh
lsof -Pni :389
```

The configuration files and what are they used for, in __/etc/ldap__.

| LDAP Folder                 | Explanation                                                     |
| --------------------------- | --------------------------------------------------------------- |
| sasl2/                      | Used for SASL authentication. Initially empty and unconfigured. |
| schema/                     | Contains the included schema and ldif files.                    |
| slapd.d/                    | The LDAP server’s configuration storage.                        |
| slapd.d/cn=config           | Contains the server configuration and directory databases.      |
| slapd.d/cn=config/cn=schema | Contains the currently loaded schemas.                          |
| ldap.conf                   | Used to define system-wide defaults for LDAP clients.           |

The actual database, that is automatically built from this configuration, is stored in /var/lib/ldap.

#### Verify the LDAP Search
To verify the ldap server is configured successfully, you can use the below command and verify that the domain entry is present.

```sh
ldapsearch -x -W -D cn=admin,dc=example,dc=com -b dc=example,dc=com -LLL
```
#### Custom Directory Creation and Directory Modification

You can set up your own directory by (writing and importing LDIF files) or with the Debian Packet Manager (dpkg). This should only be used for a first-time setup.

```sh
dpkg-reconfigure slapd
```

| Config Question                       | Answer      | Explanation                                                                          |
| ------------------------------------- | ----------- | ------------------------------------------------------------------------------------ |
| Omit OpenLDAP server configuration?   | No          | This will start the configuration wizard.                                            |
| DNS domain name:                      | example.com | Name of your directory (this will result in a BaseDN of the form dc=example,dc=com). |
| Organization name:                    | example     | Name of your organization.                                                           |
| Administrator password:               | secret      | New password for the LDAP Administrator (cn=admin,dc=example,dc=com).                |
| Database backend to use:              | HDB         | Based on Oracle Berkeley Database (BDB) but more effective.                          |
| Remove database when slapd is purged? | No          | Keep the database if OpenLDAP is uninstalled.                                        |
| Move old database?                    | Yes         | Remove the old database so that it does not interfere with the new configuration.    |

The dc=nodomain has now been replaced with dc=example,dc=com.

You can modify directory with:
1. Using the ldap-utils (ldapadd, ldapdelete, ldapmodify, …)
2. Using a graphical user interface (i.e. Apache Directory Studio)

#### Load an Additional Schemas
For sure, to use e-mail address on accounts, we need to provide the types:
- attributetype mailacceptinggeneralid to list typical aliases for the local part of the address
- attributetype maildrop for the final mail box destination used by Postfix
- objectclass postfixUser to identify users who receive mail by Postfix

> You can find the ldif file on [here](src/postfix.ldif).
         *Thanks to [Credativ](https://github.com/credativ/postfix-ldap-schema)*.

Use this line to add custom ldif file:
```sh
sudo ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/postfix.ldif
```
The new attributes are avaiable from now on.
If not, try a restart (__/etc/init.d slapd restart__).

> To load a schema with ldapadd, it has to be in LDIF format, so it must be converted first.
> - The conversion can be done with ```slapcat```. You’ll need a config file and an output directory:

 ```cd /etc/ldap/schema```
  ```mkdir ldif_output```
  ```touch schema_convert.conf```

> - The schema_convert.conf file contains the schema to be converted (and any dependencies):

  ```include /etc/ldap/schema/core.schema```

  ```include /etc/ldap/schema/cosine.schema```

  ```include /etc/ldap/schema/nis.schema```

  ```include /etc/ldap/schema/inetorgperson.schema```

  ```include /etc/ldap/schema/postfix.schema```

> - Start the conversion:

```slapcat -f schema_convert.conf -F ./ldif_output/ -n0```

```cp /etc/ldap/schema/ldif_output/cn\=config/cn\=schema/cn\=\{4\}postfix.ldif /etc/ldap/schema/postfix.ldif```
> - Finally, in the postfix-book.ldif, the following changes need to be made:
* dn: cn=postfix,cn=schema,cn=config
* cn: postfix
* Remove the lines starting from __structuralObjectClass__

>**Note:**You can find postfix.schema on [here](https://geek.co.il/articles/postfix/postfix.schema). Also other contributed schemas available on [here](https://github.com/credativ/postfix-ldap-schema)

#### Adding Users by LDIF File
Create a userimport.ldif with the following content to add a single user (the value for userPassword can be generated with slappasswd):
```sh
# Some User
dn: uniqueIdentifier=test1,ou=people,dc=example,dc=com
objectClass: organizationalPerson
objectClass: person
objectClass: top
objectClass: PostfixBookMailAccount
objectClass: extensibleObject
cn: Test1 User
givenName: Test1
mail<EMAIL_ADDRESS>mailEnabled: TRUE
mailGidNumber: 5000
mailHomeDirectory<EMAIL_ADDRESS>mailQuota: 10240
mailStorageDirectory<EMAIL_ADDRESS>mailUidNumber: 5000
sn: Test1
uniqueIdentifier: test1
userPassword: {SSHA}Ifz0oceGr1wwBP1BtBduPLTVbo6A2Qkd
```

Then add it using this command:
```sh
ldapadd -W -D "cn=admin,dc=example,dc=com" -f userimport.ldif
```
>**Note**:You can get SSHA decrypted password by using `passwd`.

#### LDAP Configuration for Test

The BaseDN for this howto is “dc=example,dc=com” and it contains two organizationalUnits:

  - “ou=people,dc=example,dc=com” for user accounts (RDN = uniqueIdentifier)
  - “ou=services,dc=example,dc=com” for service accounts (RDN = uid)
  - You’ll need accounts for postfix and dovecot

LDAP structure should be like:

### Postfix
Postfix is the default Mail Transfer Agent (MTA) in Ubuntu. It attempts to be fast and secure, with flexibility in administration. It is compatible with the MTA sendmail.

#### Postfix Installation
To install Postfix run the following command and during installation select the “No Configuration” option:
```sh
apt-get install postfix postfix-pcre postfix-ldap
```
Optionally, you can install extra tools for managing mailboxes (mutt) and sending testmails (swaks):

```sh
apt-get install mutt swaks
```

> __Note:__ If you have some access problems, set ```/etc/postfix/``` owner to ```postfix:postfix``` and its mode to ```0100```.
#### Postfix Configuration

Before starting Postfix, some configuration files need to be created and edited in __/etc/postfix__. For an example, check the files:

* [main.cf](src/main.cf) -> This is the main configuration file for Postfix
* [virtual_domains](src/virtual_domains.cf) –> Contains the domains the server takes mails for
* [ldap_virtual_recipients.cf](src/ldap_virtual_recipients.cf) –> LDAP query for recipient validation
* [ldap_virtual_aliases.cf](src/ldap_virtual_aliases.cf) –> LDAP query to get aliases and forwarding addresses
* [identitycheck.pcre](src/identitycheck.pcre) –> Regular expression to block clients that use your hostname
* [drop.cidr](src/drop.cidr) –> Contains blacklisted IP addresses

> __Note:__ You can use also ```dpkg-reconfigure postfix``` as a start.

> __Note: If any changes done in Postfix files, especially in main.cf, need to restart Postfix service!!!__

#### Starting Postfix

Start Postfix with the following command:  ```sh service postfix start ```

Check if it is running:  ```sh lsof -Pni :25 ```

Have a look at __/var/log/mail.log__ to see if there are any errors. It's helps very nice to see error that you can't see.

#### SMTP Connection Test
Use Telnet to connect to the SMTP server:  ```sh telnet <IP_ADDRESS> 25 ```. When you are connected, send an ```sh EHLO client```. You should get the following response:
>Trying <IP_ADDRESS>...

Connected to <IP_ADDRESS>.

Escape character is '^]'.

220 mail.example.com ESMTP Postfix (Ubuntu)

EHLO client

250-mail.example.com

250-PIPELINING

250-SIZE 10240000

250-ETRN

250-AUTH DIGEST-MD5 NTLM CRAM-MD5 LOGIN PLAIN

250-AUTH=DIGEST-MD5 NTLM CRAM-MD5 LOGIN PLAIN

250-ENHANCEDSTATUSCODES

250-8BITMIME

250 DSN

QUIT

221 2.0.0 Bye

#### LDAP Lookup Test

You can test the mapping with the ```postmap``` command
```sh
postmap -q<EMAIL_ADDRESS>ldap:/etc/postfix/ldap_virtual_recipients.cf
postmap -q<EMAIL_ADDRESS>ldap:/etc/postfix/ldap_virtual_aliases.cf
```
Both of the above commands should return<EMAIL_ADDRESS>
### Dovecot
Dovecot as LDA (Local Delivery Agent). This is the POP3 and IMAP server. It accepts incoming mail from Postfix and stores it in virtual mailboxes. It is connected to the LDAP for user authentication and lookups.

#### Dovecot Installation
Install Dovecot and the necessary extensions. During installation, you will be asked if a certificate should be created. If you skip this, you can add your own certificate later.
```
apt-get install dovecot-core dovecot-imapd dovecot-pop3d dovecot-lmtpd dovecot-ldap
```
#### Dovecot Configuration
The configuration is spread over several files in __/etc/dovecot/__ and __/etc/dovecot/conf.d__ and contains some basic configuration blocks, which can be included by uncommenting them. ```doveconf -n``` will print the final configuraton build up out of all snippets.

Disable unwanted protocols (IMAPS and POP3S) by setting the ports to 0 in __/etc/dovecot/conf.d/10-master.conf__. Also, set the permissions, user and group for the authentication-userdb:
```
inet_listener imaps {
    port = 0
    #port = 993
    #ssl = yes
}
inet_listener pop3s {
    port = 0
    #port = 995
    #ssl = yes
}
unix_listener auth-userdb {
    mode = 0600
    user = vmail
    group = vmail
}
```
Dovecot can use LDAP as a password database for authentication as well as a user database for information like the maildir location, and the UID and GID of the user. To activate LDAP as a password and user database, enable it in __/etc/dovecot/conf.d/10-auth.conf__ and disable __auth-system.conf.ext__.
```
auth_mechanisms = plain login
#!include auth-system.conf.ext
!include auth-ldap.conf.ext
```
__/etc/dovecot/conf.d/auth-ldap.conf.ext__ should contain the declaration for __passdb__ and __userdb__. Set the LDAP-related parameters in __/etc/dovecot/dovecot-ldap.conf.ext__:
```
hosts = <IP_ADDRESS>
dn = uid=dovecot,ou=services,dc=example,dc=com
dnpass = secret
ldap_version = 3
base = ou=people,dc=example,dc=com
user_attrs = mailHomeDirectory=home,mailUidNumber=uid,mailGidNumber=gid,mailStorageDirectory=mail
user_filter = (&(objectClass=PostfixBookMailAccount)(uniqueIdentifier=%n))
pass_attrs = uniqueIdentifier=user,userPassword=password
pass_filter = (&(objectClass=PostfixBookMailAccount)(uniqueIdentifier=%n))
default_pass_scheme = CRYPT

passdb {
  driver = ldap

  # Path for LDAP configuration file, see example-config/dovecot-ldap.conf.ext
  args = /etc/dovecot/dovecot-ldap.conf.ext
}
userdb {
  driver = ldap
  args = /etc/dovecot/dovecot-ldap.conf.ext

  # Default fields can be used to specify defaults that LDAP may override
  #default_fields = home=/home/virtual/%u
}
```
Activate logging in __/etc/dovecot/conf.d/10-logging.conf__:
```
log_path = syslog
syslog_facility = mail
auth_debug = yes
```
The location of the Maildir is set in __/etc/dovecot/conf.d/10-mail.conf__. The user's homedirectory will be read from LDAP, and '/mailbox' will be appended with the following configuration:
```
maildir:~/mailbox
```

Add a system user and group named vmail with uid and gid 5000:
```
addgroup --system --gid 5000 vmail
adduser --system --home /srv/vmail --uid 5000 --gid 5000 --disabled-password --disabled-login vmail
```
Make sure that __/srv/vmail__ has been created.

#### Starting Dovecot
Start Dovecot with ```service dovecot start``` and check if it is running with  ```lsof -Pni — port 110``` and __143__ should be open. If it isn’t running, check the logfiles (mail.log and syslog). If there are no errors in the log, start Dovecot in foreground mode to have the errors printed to the console:  dovecot -F.

#### IMAP Connection & Authentication Test
Use Telnet to connect to Dovecot’s IMAP Server: ```telnet <IP_ADDRESS> 143```. Then send a login request ( ```1 login<EMAIL_ADDRESS>secret```) to see if authentication (plain) is working:

```Trying <IP_ADDRESS>...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot (Ubuntu) ready.
1 login<EMAIL_ADDRESS>secret
1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in
2 logout
* BYE Logging out
```
#### Postfix Integration
Activate Dovecot Deliver in Postfix by adding these lines to __/etc/postfix/master.cf__:
```
dovecot   unix  -       n       n       -       -       pipe
        flags=ODRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -e -f ${sender} -d ${recipient}
```
Set the postmaster address in __/etc/dovecot/conf.d/15-lda.conf__:
```
postmaster_address =<EMAIL_ADDRESS>```
Restart Dovecot and Postfix:
```
service dovecot restart
service postfix restart
```
#### Dovecot Deliver Test
Send a testmail to<EMAIL_ADDRESS>using one of the following methods:

```
echo Testmail | sendmail -f<EMAIL_ADDRESS><EMAIL_ADDRESS>```
```
swaks --from<EMAIL_ADDRESS>--to<EMAIL_ADDRESS>--server <IP_ADDRESS>:25
```

```
telnet <IP_ADDRESS> 25

Trying <IP_ADDRESS>...
Connected to localhost.
Escape character is '^]'.
220 mail.example.com ESMTP Postfix (Ubuntu)
EHLO test.local
250-mail.example.com
250-PIPELINING
250-SIZE 10240000
250-ETRN
250-AUTH DIGEST-MD5 NTLM CRAM-MD5 LOGIN PLAIN
250-AUTH=DIGEST-MD5 NTLM CRAM-MD5 LOGIN PLAIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
MAIL<EMAIL_ADDRESS>250 2.1.0 Ok
RCPT<EMAIL_ADDRESS>250 2.1.5 Ok
DATA
354 End data with <CR><LF>.<CR><LF>
Testmail
.
250 2.0.0 Ok: queued as 51586A1146
QUIT
221 2.0.0 Bye
```
Check the log /var/log/mail.log and Alice’s mailbox:  ```mutt -f<EMAIL_ADDRESS>```

#### Log Viewing
Postfix sends all log messages to __/var/log/mail.log__. However, error and warning messages can sometimes get lost in the normal log output so they are also logged to __/var/log/mail__.err and __/var/log/mail.warn__ respectively.

**Note that**
Files and folders containing passwords:
- /etc/postfix/ldap_virtual_aliases.cf

- /etc/postfix/ldap_virtual_recipients.cf

- /etc/dovecot/dovecot-ldap.conf.ext

- /etc/saslauthd.conf

- /var/www/html/roundcube/config/config.inc.php

Folders containing private keys:
- /etc/postfix/certs

- /etc/dovecot/private

- /etc/ssl/private (possibly for your Apache key)

>For some reason, I need to change ```/etc/dovecot/dovecot.conf``` file to get rid of errors:
>```
>service stats {
>    unix_listener stats-reader {
>        user = vmail
>        group = vmail
>        mode = 0666
>    }
>
>    unix_listener stats-writer {
>        user = vmail
>        group = vmail
>        mode = 0666
>    }
>}
>```

Source:

- Mastering CentOS 7 Linux Server

repository: dorukalioglu/ldap

repo_name: dorukalioglu/ldap


text:
---
layout: post
title: "Single input, multiple outputs"
category: Text
tags:
- pandoc
- text
- conversion
- LaTeX
---

We've briefly discussed Pandoc now.
It bills itself as the "Universal Document Converter."
This is reasonably true, but it might require some creative combinations of switches within Pandoc commands as well as multiple commands strung together or intermediate commands to get the desired output.

The benefit of troubleshooting and understanding this process is that once we do, we can more easily optimize our conversions and automate them.
We'll talk more about this as we go forward.
<excerpt/>

# Source to output conversion

It is possible to use GUI tools to create and convert documents.
Support is somewhat limited, but in LibreOffice, we can at least create a PDF from our ODT and DOCX files.

We can also manipulate the styles of the headers and other structural elements that we have assigned using Markdown in our GUI editors.

One convenient effect of starting with plaintext marked up with Markdown is that we have those structural elements when we convert them into another format and then edit them elsewhere. It is certainly possible to start in the GUI editor and define the same things, but after becoming acquainted with Markdown, it should feel somewhat more burdensome to use the GUI. Arguably, it is. There is a great deal more that goes into a DOCX or an ODT file, structurally, than in a plaintext file with Markdown in it.

We also have the disadvantage of only being able to operate on thost files in limited ways on headless or remote systems.

If we keep plaintext at the core of our workflows and GUI editors toward the periphery, we will be served in the end as we will always have access to our work, on any system, without any huge barriers to editing and changing.

## Commands

We're going to practice some conversions using Pandoc today. We will also work in groups.

First, we need to get some files.

Fork and clone this repository into your CodeAnywhere container:

https://github.com/inls161/pandoc-practice

Once you have the files in your CodeAnywhere container, I will show you some things in class and then in your groups you will answer and mark up the `example.md` file using the instructions in the file.

***When you are finished with the questions in file:***

1. I want you to change the name of the file to your GitHub username.
2. You will then work as a group to convert the file to HTML, DOCX, and ODT formats, per the instructions in class. I also want you to open the files on your lab computers so you can see what you have done.
3. Then I want you to add, commit, and push your changes.
4. Finally, you will create a pull request in GitHub to get these files back into my original repository.

## Basic Pandoc commands

{% marginnote 'pandoc-commands' 'All Pandoc commands are documented here: http://pandoc.org/README.html

A good set of example commands exists here: http://pandoc.org/demos.html' %}

Convert a Markdown file to HTML:

`pandoc -o example.html example.md`

Pandoc reads the filetype from the extension in normal usage.
If you want to convert a file directly from a URL, you will have to specify the filetype, like this:

`pandoc -f html -t markdown http://inls161.johndmart.in/syllabus/`

You can make sure that certain things, like quotes and em-dashes, get read and formatted propery by specifying the "Smart" switch (a capital `-S` or `--smart`):

`pandoc -S -o example.html example.md`

There are a host of other commands in the documentation. Be sure to try them out.

## Specific file commands

Convert your markdown to HTML:

`pandoc -o example.html exampld.md`

If you wish to convert to a DOCX or ODT file:

`pandoc -o example.docx example.md`

`pandoc -o example.odt example.md`

If you wish to convert between two different word processor filetypes, we might have to get a little creative.
We learned in class that if we issue the following command, we get errors related to file encoding and the conversion will not work.

`pandoc -o example.docx example.odt`

If, however, we add an intermediary step, say through HTML, we can get the output that we want. Try it like this instead:

`pandoc -o example.docx example-tmp.html && pandoc -o example.odt example-tmp.html`

This preserves the formatting and extracts the text from the DOCX as an HTML file and then converts that HTML into ODT.
We do not have the weird encoding errors this way, and we don't have to mess with pipes.

Filter a document through a template file:

`pandoc -S --reference-docx=FILE -o example.docx example.md`

In the above command, you need to specify the location of the template file.
If it is a file called `template.docx` and is located in the same directory as your Markdown source, then the command will be:

`pandoc -S --reference-docx=./template.docx -o example.docx example.md`

You can also use an ODT or OTT for reference:

`pandoc -S --reference-odt=./template.ott -o example.odt example.md`

# For tomorrow

Tomorrow, we are going to learn one more output format and then learn how to script all of our outputs together so that we can save ourselves time.

I would like you to go through the Writing Shell Scripts tutorial by William Shotts for tomorrow.{% sidenote 'shell-scripts' 'Shotts, William, Jr. “Writing Shell Scripts.” LinuxCommand.org. Accessed July 6, 2016. http://linuxcommand.org/lc3_writing_shell_scripts.php.' %}
This will show you the basics of scripting.
The scripts that we will write will be very, very simple, but it is good to have looked over this before we start.


text:
- [Introduction](#introduction)
- [Requirements/Setup](#requirementssetup)
- [Usage](#usage)
  * [Compiling random forest/decision tree](#compiling-random-forestdecision-tree)
  * [Compiling a tensorflow model](#compiling-a-tensorflow-model)
  * [Compiling an ONNX model](#compiling-an-onnx-model)
  * [Compiling and Running Models in Networks Directory Automatically](#compiling-and-running-models-in-networks-directory-automatically)
      - [Running manually (non-tmux-mode)](#running-manually-non-tmux-mode)
  * [Compiling and Running Models in Networks Directory Manually](#compiling-and-running-models-in-networks-directory-manually)
- [Directory structure](#directory-structure)
- [Preprocessing images and running inference on ImageNet validation dataset](#preprocessing-images-and-running-inference-on-imagenet-validation-dataset)

# Introduction
This folder contains the code for Athos - an end-to-end compiler from TensorFlow to a variety of secure computation protocols.

# Requirements/Setup
If you used the `setup_env_and_build.sh` script the below would already have been installed in the `mpc_venv` environment. We require the below packages to run Athos.
- python3.7
- TensorFlow 1.15
- Numpy
- pytest, pytest-cov (For running tests)
- onnx, onnx-simplifier

Athos also makes use of the EzPC compiler internally (please check `../EzPC/README.md` for corresponding dependencies).

# Usage
Please source the virtual environment in `mpc_venv` if you used the `setup_env_and_build.sh` script to setup and build.

`source mpc_venv/bin/activate`

## Compiling random forest/decision tree
Use `CompileRandomForests.py` according to the readme in [Athos/RandomForests](https://github.com/mpc-msri/EzPC/tree/master/Athos/RandomForests)

## Compiling a tensorflow model
The `CompileTFGraph.py` script can compile tensorflow models (v1.15). You can dump your tensorflow model as a frozen graph. Run [convert_variables_to_constants](https://www.tensorflow.org/api_docs/python/tf/compat/v1/graph_util/convert_variables_to_constants) on your model graph and then dump the output graph_def as a protobuf (see `dump_graph_def_pb` in `CompilerScripts/tf_graph_io.py`). Once you have the model.pb file simply do:
```
python CompileTFGraph.py --config config.json --role server
```
See `python CompileTFGraph.py --help` for additional details on the config.json parameters. A sample config could be:
```
{
  "model_name": "model.pb",
  "output_tensors": [ "output1" ],
  "target": "SCI",
  "backend": "OT",
}
```
You will see the output messages of the compiler and a `model_SCI_OT.out` binary will be generated. You will also see this in the output:
```
Use as input to server (model weights): model_input_weights_fixedpt_scale_12.inp.
Share client.zip file with the client
```
Use the `model_input_weights_fixedpt_scale_12.inp` file as input for the server party. The additional client.zip file contains a version of the model without model weights and additionally contains the config file. This zip file should be sent to the client and after unzipping, they can compile the model with:
```
python CompileTFGraph.py --config model.config --role client
```
For model input you can create a random input using `CompilerScripts/create_tf_input.py` or pass your actual input as a numpy array to the `dumpImageDataInt` function in `TFCompiler/DumpTFMtData.py`. For both scripts you need to pass the scaling factor for conversion of floating point to fixed point (we use 12 for ResNet). Refer to [Running manually (non-tmux-mode)](#running-manually-non-tmux-mode) on how to run the MPC protocol or this [blog post](https://pratik-bhatu.medium.com/privacy-preserving-machine-learning-for-healthcare-using-cryptflow-cc6c379fbab7) for a more detailed walkthrough.

## Compiling an ONNX model
Similar to how we compile tensorflow graphs, we have a `CompileONNXGraph.py` script that can compile onnx models. The usage is exactly the same as the `CompileTFGraph.py` script.

### Supported nodes

Some of the supported nodes in ONNX models are:

```Cast
Pad
Concat
HardSigmoid ( Only in SCI )
Relu
Div
Add
Sub
Mul
Clip ( Only in SCI )
Gather
ArgMax
Gemm
Constant
Transpose
Split
ReduceMean
MatMul
BatchNormalization
Unsqueeze
Reshape
Flatten
Conv
MaxPool
AvgPool
AveragePool
GlobalAveragePool
ConvTranspose
```

There are some also additional limitations in some of the nodes. The compiler will exit with information about the limitation when the model with an unsupported node is compiled.

## Compiling and Running Models in Networks Directory Automatically
The `CompileSampleNetworks.py` script can compile and optionally run models in the Networks directory like ResNet-50, DenseNet, SqueezeNet, etc..
To compile and run ResNet with the Porthos semi-honest 3PC protocol we do:

```python CompileSampleNetworks.py --config Networks/sample_network.config```

The script takes a config file as input. The contents of the config are:
```
{
  "network_name":"ResNet",
  "target":"PORTHOS",
  "run_in_tmux": true
}
```
- *network_name*: Can be any network in the Networks directory.
- *target*: This is the secure protocol the model will run in. The possible values are:
	- **PORTHOS**: The semi-honest 3PC protocol.
	- **SCI**: The semi-honest 2PC protocol in SCI.
	- **CPP**: A non-secure debug backend which outputs plain C++ to test for correctness.
- *run_in_tmux*: If true, the script spawns a tmux session to run the network. There is a terminal pane for each party.
You can modify the config file according to which network and backend you want to compile for. See ```python CompileSampleNetworks.py --help``` for more information about the parameters of the config file.

**Output:**
After connecting to the session with ```tmux a -t ResNet``` you should see output similar to the following after the computation is complete. Numbers will vary based on the specs of your machine.

| | |
|-|-|
|-------------------------------------------------------
                  **ResNet results [Client]**
-------------------------------------------------------
Model outputs:
MPC PORTHOS (3PC) output:        249
Tensorflow output:               249

Execution summary for Client:
Communication for execution, P0: 2377.13MB (sent) 1825.32MB (recv)
Peak Memory Usage:               432156 KB (.41GB)
Total time taken:                70.50 seconds
Total work time:                 68.57 seconds (97.27%)
Time spent waiting:              1.91 seconds (2.72%)
Time taken by tensorflow:        0.31 seconds |
|-------------------------------------------------------
                  **ResNet results [Server]**
-------------------------------------------------------
Execution summary for Server:
Communication for execution, P1: 2377.13MB (sent) 2155.96MB (recv)
Peak Memory Usage:               432428 KB (.41GB)
Total time taken:                70.60 seconds
Total work time:                 68.77 seconds (97.42%)
Time spent waiting:              1.81 seconds (2.57%)   |-------------------------------------------------------
                  **ResNet results [Helper]**
-------------------------------------------------------
Execution summary for Helper:
Communication for execution, P2: 2113.81MB (sent) 2886.8MB (recv)
Peak Memory Usage:               427508 KB (.40GB)
Total time taken:                70.53 seconds
Total work time:                 64.06 seconds (90.83%)
Time spent waiting:              6.46 seconds (9.16%)

#### Running manually (non-tmux-mode)

If run_in_tmux is false and you want to run the network manually, you will need the following files that are generated by the script in the `Networks/ResNet` directory:
- *ResNet_PORTHOS.out*:		binary of the compiled network.
- *model_input_scale_12.inp*: 	image input to the model.
- *model_weights_scale_12.inp*: model weights.

Running it manually will not give you a neat summary as shown above but will print the computed output on the client terminal.

To run the network in **3PC mode (PORTHOS)**, open 3 terminals and do the following for each party:

- Party 0 [Client]:

	``` ./Networks/ResNet/ResNet_PORTHOS.out 0 ../Porthos/files/addresses ../Porthos/files/keys < model_input_scale_12.inp" ```
- Party 1 [Server]:

	``` ./Networks/ResNet/ResNet_PORTHOS.out 1 ../Porthos/files/addresses ../Porthos/files/keys < model_weights_scale_12.inp" ```
- Party 2 [Helper]:

	``` ./Networks/ResNet/ResNet_PORTHOS.out 1 ../Porthos/files/addresses ../Porthos/files/keys" ```

To run the network in **2PC mode (SCI)**, open 2 terminals and do the following for each party:

- Party 0 [Server]:

	``` ./Networks/ResNet/ResNet_SCI_OT.out r=1 p=12345 < model_weights_scale_12.inp" ```
- Party 1 [Client]:

	``` ./Networks/ResNet/ResNet_SCI_OT.out r=2 ip=<IP_ADDRESS> p=12345 < model_input_scale_12.inp" ```

To run the network in **CPP mode (1PC-debug-non-secure)**, open a terminal and do the following:
- ``` ./Networks/ResNet/ResNet_CPP.out < <(cat model_input_scale_12.inp model_weights_scale_12.inp) ```

## Compiling and Running Models in Networks Directory Manually
To better understand what the `CompileSampleNetworks.py` script is doing under the hood, we can step through each step manually. Here we provide an example on how to use Athos to compile TensorFlow based ResNet-50 code to Porthos semi-honest 3PC protocol and subsequently run it. The relevant TensorFlow code for ResNet-50 can be found in `./Networks/ResNet/ResNet_main.py`.
- Refer to `./Networks/ResNet/README.md` for instructions on how to download and extract the ResNet-50 pretrained model from the official TensorFlow model page.
- `cd ./Networks/ResNet && python3 ResNet_main.py --runPrediction True --scalingFac 12 --saveImgAndWtData True && cd -`
Runs the ResNet-50 code written in TensorFlow to dump the metadata which is required by Athos for further compilation.
This command execution should result in 2 files which will be used for further compilation - `./Networks/ResNet/graphDef.mtdata` and `./Networks/ResNet/sizeInfo.mtdata`. In addition, the image and the model are also saved in fixed-point format, which can be later input into the compiled code - `./Networks/ResNet/model_input_scale_12.inp` which contains the image and `./Networks/ResNet/model_weights_scale_12.inp` which contains the model weights.
- The next step is to perform the compilation itself. The compilation script internally makes use of the `ezpc` executable. So, before continuing please ensure that you have built `ezpc` (please check the `../EzPC/README.md` for further instructions on that).
- Once EzPC has been built, run this to compile the model to Porthos - `./CompileTF.sh -b 64 -s 12 -t PORTHOS -f ./Networks/ResNet/ResNet_main.py`. This should result in creation of the file - `./Networks/ResNet/ResNet_main_64_porthos0.cpp`.
- `cp ./Networks/ResNet/ResNet_main_64_porthos0.cpp ../Porthos/src/main.cpp`
Copy the compiled file to Porthos.
- `cd ../Porthos && make clean && make -j`
- Finally run the 3 parties. Go to the porthos directory and open 3 terminals and run the following in each for the 3 parties.
`./party0.sh < ../Athos/Networks/ResNet/ResNet_img.inp` ,
`./party1.sh < ../Athos/Networks/ResNet/ResNet_weights.inp` ,
`./party2.sh`.
Once the above runs, the final answer for prediction should appear in the output of party0, the client inputting the image. For the sample image, this answer should be 249 for ResNet and 248 for DenseNet/SqueezeNet.

Instructions on how to run the particular TensorFlow model in `./Networks` can vary. Please refer to the appropriate readme in each model folder to get more insights. But once that is done, the further compilation commands are the same.

# Directory structure
The codebase is organized as follows:
- `HelperScripts`: This folder contains numerous helper scripts which help from automated setup of ImageNet/CIFAR10 dataset to finding accuracy from output files. Please refer to each of the scripts for further instructions on how to use them.
- `Networks`: This folder contains the code in TensorFlow of the various benchmarks/networks we run in CrypTFlow. Among other networks, it includes code for ResNet, DenseNet, SqueezeNet for ImageNet dataset, SqueezeNet for CIFAR10 dataset, Lenet, Logistic Regression, and a chest x-ray demo network.
- `SeeDot`: This contains code for SeeDot, a high-level intermediate language on which Athos performs various optimizations before compiling to MPC protocols.
- `TFCompiler`: This contains python modules which are required by Athos for compilation of tensorflow models to MPC protocols.
- `ONNXCompiler`: This contains python modules which are required by Athos for compilation of ONNX models to MPC protocols.
- `TFEzPCLibrary`: This contains library code written in EzPC for the TensorFlow nodes required during compilation.
- `CompileTF.sh`: The Athos compilation script. Try `./CompileTF.sh --help` for options.
- `CompileTFGraph.py`: The Athos compilation script for tensorflow models. Try `python CompileTFGraph.py --help` for options.
- `CompileONNXGraph.py`: The Athos compilation script for ONNX models. Try `python CompileONNXGraph.py --help` for options.
- `Paths.config`: This can be used to override the default folders for EzPC and Porthos.
- `CompilerScripts`: This folder contains scripts used for processing and compiling dumped models.

# Preprocessing images and running inference on ImageNet validation dataset
- First setup the ImageNet validation dataset using the script provided in `./HelperScripts/Prepare_ImageNet_Val.sh`. This sets up the ImageNet validation dataset in the folder - `./HelperScripts/ImageNet_ValData`.
- Each of the network folders - `./Networks/ResNet`, `./Networks/DenseNet` and `./Networks/SqueezeNetImgNet` is provided with these folders:
	* `PreProcessingImages`: This folder contains code for preprocessing the images. Code borrowed from the appropriate repository from where the model code is taken
and modified for our purposes (check the apt network folder for more details on the source of the model).
	* `AccuracyAnalysisHelper`: This contains further scripts for automating ImageNet dataset preprocessing and inference. Check the apt scripts for more information.

repository: mpc-msri/EzPC

repo_name: mpc-msri/EzPC


text:
# State Table

State table contains a list of states, a description, the inputs and the next
state.

Each state can be one of four types.

| Type | Description |
| :--- | :---------- |
| MSG  | Display message |
| INPUT | Process input |
| WAIT  | Specific input type awaiting a specific key |
| ACTION | Do something |

INPUT and WAIT states typically require a MSG state prior to entry into these
state types.

ACTION states typically

| #   | Description                                            | Input | Next  |
| :-- | :----------------------------------------------------- | :---- | :---- |
| u   | default                                                |       | 0     |
| 0   | MSG:Select Mode                                        |       | 1     |
| 1   | INPUT:Mode                                             | D     | 2     |
|     |                                                        | A     | 8     |
| 2   | MSG:Division Entry                                     |       | 3     |
| 3   | INPUT:Number of divs                                   |       |       |
|     |   --> Expects 2 or greater                             | #<2   | 2     |
|     |                                                        | >=2   | 4     |
| 4   | MSG:Run Cutter                                         |       |       |
|     |   --> if(division == totalDivisions)                   | CS    | 7     |
|     |                                                        | CS    | 5     |
| 5   | WAIT:CS to rotate                                      | CS    | 6     |
| 6   | ACTION:rotate                                          |       | 4     |
| 7   | WAIT:CS for final pass                                 | CS    | 0     |
| 8   | MSG:Angle Entry                                        |       | 9     |
| 9   | INPUT:Angle                                            |       |       |
|     |   --> if(angle == 0) go to main menu                   | #=0   | 0     |
|     |   --> if(angle > 0 )                                   | #>0   | 10    |
| 10  | ACTION:rotate n degrees                                |       | 9     |

repository: carneeki/FUCAN-spintroller

path: /State Table.md

filename: State Table.md

repo_name: carneeki/FUCAN-spintroller


text:
This is a benchmark for finding a
[heap-buffer-overflow bug](https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=925) in
[Little-CMS](https://github.com/mm2/Little-CMS).

Note that, in OSS-Fuzz, this bug was first found with [AFL](http://lcamtuf.coredump.cx/afl/).

The following error can be found within 30 minutes of fuzzing, from the provided seed.

```
==27232==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60800011b68c at pc 0x00000057c11a bp 0x7ffd7544b130 sp 0x7ffd7544b128
READ of size 4 at 0x60800011b68c thread T0
    #0 0x57c119 in TetrahedralInterpFloat BUILD/src/cmsintrp.c:642:22
    #1 0x599c56 in _LUTeval16 BUILD/src/cmslut.c:1330:14
    #2 0x51a13a in CachedXFORM BUILD/src/cmsxform.c:525:17
    #3 0x512b8d in cmsDoTransform BUILD/src/cmsxform.c:189:5
    #4 0x4ea37c in LLVMFuzzerTestOneInput cms_transform_fuzzer.c
```

Generally, the above error is found. However, the [following error](https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=931) is also present.

```
==96256==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60800000048c at pc 0x00000057bfea bp 0x7ffd1790c010 sp 0x7ffd1790c008
READ of size 4 at 0x60800000048c thread T0
    #0 0x57bfe9 in TetrahedralInterpFloat BUILD/src/cmsintrp.c:642:22
    #1 0x59a1f1 in _LUTevalFloat /usr/local BUILD/src/cmslut.c:1356:15
    #2 0x54e591 in XFormSampler16 BUILD/src/cmsopt.c:423:5
    #3 0x593e77 in cmsStageSampleCLut16bit BUILD/src/cmslut.c:797:14
    #4 0x54cdbf in OptimizeByResampling BUILD/src/cmsopt.c:734:10
    #5 0x54a74f in _cmsOptimizePipeline BUILD/src/cmsopt.c:1942:17
    #6 0x51521f in AllocEmptyTransform BUILD/src/cmsxform.c:819:15
    #7 0x5140a0 in cmsCreateExtendedTransform BUILD/src/cmsxform.c:1075:13
    #8 0x516ce2 in cmsCreateMultiprofileTransformTHR BUILD/src/cmsxform.c:1175:12
    #9 0x516ce2 in cmsCreateTransformTHR BUILD/src/cmsxform.c:1216
    #10 0x516ce2 in cmsCreateTransform BUILD/src/cmsxform.c:1226
    #11 0x4ea02c in LLVMFuzzerTestOneInput cms_transform_fuzzer.c:31:30
```

repository: DalavanCloud/fuzzer-test-suite

repo_name: DalavanCloud/fuzzer-test-suite


text:
---
layout:     post
title:      "Online Detection of Unusual Events in Videos via Dynamic Sparse Coding"
subtitle:   "论文笔记-2011-CVPR-139"
date:       2016-09-17
author:     "icbcbicc"
header-img: "img/post-bg-14.jpg"
tags: []
---

# Online Detection of Unusual Events in Videos via Dynamic Sparse Coding
Authors: *Bin Zhou, Li Fei-Fei, Eric P. Xing*
[PDF](http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=5995524)

<br>

## Introduction

####

url: https://raw.githubusercontent.com/icbcbicc/icbcbicc.github.io/b0d303f6fd4f19dab351875d775222a8eec5680f/_posts/2016-09-18-Online Detection of Unusual Events in Videos via Dynamic Sparse Coding.md


text:
# Security Policy

**Note**: Please do not submit security reports via Github issues.

To read the most up to date version of our security policy, including
directions for submitting security vulnerabilities, please visit
https://pypi.org/security/.


text:
# CodePipeline Flood.IO Action

![Architecture][architecture]

This project provides an integration for [Flood.IO](https://flood.io) in [AWS CodePipeline](https://aws.amazon.com/documentation/codepipeline/).

Enabling automated load testing as an action within a pipeline. You may pass "flood files" (Jmeter, Selenium, etc.) from a pipeline artifact, as well as optionally define parameter overrides using artifact attribute keys (as described below).

In-order to ensure your Pipeline has necessary permissions to invoke a Lambda function, follow the [setup steps here](http://docs.aws.amazon.com/codepipeline/latest/userguide/actions-invoke-lambda-function.html).

The flood test is monitored by an AWS Step Functions State Machine (as described above).

An example action definition inside a CodePipeline CloudFormation resource definition:

```yaml
- Name: Testing
  Actions:
    - Name: Flood
      RunOrder: 1
      InputArtifacts:
        - Name: SourceOutput
        - Name: TestStackOutput
      ActionTypeId:
        Category: Invoke
        Owner: AWS
        Version: 1
        Provider: Lambda
      OutputArtifacts:
        - Name: BuildOutput
      Configuration:
        FunctionName: !ImportValue FloodTestingFunctionName
        UserParameters: !Sub |
          {
            "Parameters": {
              "SOME_VAL": "${PipelineParameter}",
              "API_URL": {
                "Fn::GetParam": ["TestStackOutput", "Outputs.json", "ApiUrl"]
              }
            },
            "Flood": {
              "tool": "jmeter"
            },
            "Files": [
              "SourceOutput::tests/load-test-a.jmx",
              "SourceOutput::tests/load-test-b.jmx"
            ],
            "Grids": [{
              "region": "us-east-1",
              "instance_quantity": 4,
              "stop_after": 120
            }]
          }
```

For a full definition of what you can define inside the `UserParameters` key, see [the JSON Schema file](/functions/pipeline/pipeline/validate/schema.js).

[architecture]: /.github/images/Flood.png

repository: davidkelley/aws-pipeline-flood

repo_name: davidkelley/aws-pipeline-flood


text:
# Aura Executable
Aura Executable files (.aexe) can be executed on Aura Operating System!

# Build your executable from C
You will need gcc. Just type "make" in your console within this folder.

# Current features
- printf (to print a char[])
- fgets (to read a line and return it in a char[])
- clear (to clear the Aura shell)
- setcursor (to set the position of the cursor)
- resetcursor (if you want to use setcursor you MUST add this method after each printf...)
- Broken_memcpy (The memcpy function is used to copy a block of data from a source address to a destination address.)
- Broken_memset (memset is used to fill a block of memory with a particular value.)
- Broken_memcmp (Compares the first num bytes of the block of memory pointed by ptr1 to the first num bytes pointed by ptr2, returning zero if they all match or a value different from zero representing which is greater if they do not.)
- Broken_memchr(Searches within the first num bytes of the block of memory pointed by ptr for the first occurrence of value (interpreted as an unsigned char), and returns a pointer to it.)
- Broken_strcat (Appends a copy of the source string to the destination string. The terminating null character in destination is overwritten by the first character of source, and a null-character is included at the end of the new string formed by the concatenation of both in destination.)

# How is it made?
We were inspired by the MSDOS COM files. Indeed, each function uses an interrupt to call the Aura's system calls.

To communicate between Aura Operating System and the compiled C file, the variables are saved into registers (EDI, ESI).

The EAX register is how Aura knows (after an Aura System Call: "int 0x48") what function is called. For example, if EAX is 0x01 it's the print function, if it's 0x02 it's to clear the screen etc.

repository: kozit/Aura-Executable

repo_name: kozit/Aura-Executable


text:
# EEG2OSC

EEG2OSC is simple application written in Objective-C for two-way communication with [Emotiv EPOC](http://emotiv.com/epoc/) using familiar OSC protocol.

EEG2OSC was prepared for [Sensorium](http://sensorium.com.pl) - audio installation by [Rafał Zapała](http://www.zapala.com.pl), more info: [http://treesmovethemost.com/2014-10-10-sensorium](http://treesmovethemost.com/2014-10-10-sensorium).

## Outgoing OSC messages

`/EEG/ED_AF3`, `/EEG/ED_F7`, `/EEG/ED_F3`, `/EEG/ED_FC5`, `/EEG/ED_T7`, `/EEG/ED_P7`, `/EEG/ED_O1`, `/EEG/ED_O2`, `/EEG/ED_P8`, `/EEG/ED_T8`, `/EEG/ED_FC6`, `/EEG/ED_F4`, `/EEG/ED_F8`, `/EEG/ED_AF4` - raw data from EEG (requires research edition of Emotiv SDK)

`/quality/ED_AF3`, `/quality/ED_F7`, `/quality/ED_F3`, `/quality/ED_FC5`, `/quality/ED_T7`, `/quality/ED_P7`, `/quality/ED_O1`, `/quality/ED_O2`, `/quality/ED_P8`, `/quality/ED_T8`, `/quality/ED_FC6`, `/quality/ED_F4`, `/quality/ED_F8`, `/quality/ED_AF4` - electrode connection quality

`/affectiv/engagement`, `/affectiv/excitement-long`, `/affectiv/excitement-short`, `/affectiv/frustration`, `/affectiv/meditation` - affective data (`0.0` - `1.0` values)

`/expressiv/blink`, `/expressiv/wink-left`, `/expressiv/wink-right`, `/expressiv/look-left`, `/expressiv/look-right` - expressiv data (trigger)

`/gyro` - x/y gyroscope (two integer values)

`/cognitiv/action` - recognized action with power, currently supported actions: "neutral", "push", "lift" (action and `0.0` - `1.0` value)

`/cognitiv/event` - training events: "started", "succeeded", "failed", "completed", "rejected"

## Incoming OSC messages

`/train/neutral` - train neutral state

`/train/lift` - train "lift" action

`/train/push` - train "push" action

`/train/accept` - accept training

`/train/reject` - reject training

## Working with training actions

Training is the trickiest part, example workflow for training "neutral" action:

1. Send `/train/neutral` to EEG2OSC
2. Wait for `/cognitiv/event` "started"
3. Wait for `/cognitiv/event` "succeeded"
4. Send `/train/accept`
5. Wait for `/cognitiv/event` "completed"
6. Wait for `/cognitiv/action` "neutral **value**", where **value** is between `0.0` and `1.0`

Training for push and lift looks the same, but remember to always train "neutral" first.


text:
# PlaceClone

A simple Reddit Place like game. Can be played from browser.

## Deployment
First you need Node.js and npm installed on your machine. Then copy the files and run

```
npm install
```
in both 'core' and 'game' directories. Then run

```
npm start
```
in separate terminals (first in 'core', second in 'game'). Now you can open localhost:5000 in your browser and try the game.

## Built With

* [Node.js](https://nodejs.org/) - The base
* [Express](http://expressjs.com/) - Node.js framework
* [Socket.IO](https://socket.io/) - Used for realtime communication

## License

This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details


text:
# Creating data visualizations in nteract

They say a picture is worth a thousand words and that's certainly the case when it comes to data. Data visualizations are a great way to communicate large amounts of information clearly and efficiently. This guide will explore different ways to create visualizations in the nteract app.

### Plotting with matplotlib and seaborn

Matplotlib is one of the most popular plotting tools in the open source ecosystem. It's used in a variety of academic and industry contexts and is a great starting point for anyone looking to dive into data visualization.

To get started with matplotlib, we'll use another nteract example notebook. Before we get started, make sure you have the dependencies needed for this example notebook installed. The installation process will vary depending on your Python setup, but here is an example of the dependencies that need to be installed assuming a global Python3 installation.

```
$ python3 -m pip install pandas requests matplotlib seaborn
```

Once you've installed these dependencies on your machine, you'll need to open the example notebook by navigating to **FIle > Open Example Notebook > python > Glean the Download Statistics for nteract Desktop**. Navigate to **Cell > Run All** to execute all the cells in this notebook.

This notebook uses the GitHub API to fetch download statistics for OS-specific versions of the nteract desktop and renders the number of downloads per release. If you scroll to the last cell in the notebook, you'll see the following.

![A matptlotlib visualization within an nteract notebook](https://cldup.com/M-dPmjYXgL.png)

The plot is rendered as a static image. You can save the image to your desktop or laptop. You can also export this notebook as a PDF with this image inline.

### Visualizing with Plotly

In addition to rendering with matplotlib and other language-specific plotting tools, nteract also provides a custom renderer for visualizations rendered using [Plotly's](https://plot.ly/) open source visualization tools. We'll use another example notebook to explore this, but first, make sure that you have the `plotly` Python package installed in your setup.

```
$ python3 -m pip install plotly
```

Next, navigate to **File > Open Example Notebook > python > Gallery of Plotly Plots**. This notebook defines a `plotize` function that takes the data and layout of a visualization and renders it inline using the Plotly transform.

![A Plotly visualization](https://cldup.com/dlhVzlypSi.png)

Within this transform, you can zoom in and out of a visualization, full screen it, save the image, and more.

### Exploring with the Data Explorer

Last but not least, nteract ships with the nteract Data Explorer, a tool that takes all the pain out of creating data visualizations in your workflow. Instead of writing your own visualization code, the data explorer provides several visualization and customization options.

Let's try the Data Explorer out by using one of the example notebooks. Navigate to **File > Open Example Notebook > python > Try the Data Explorer**. The last cell of the notebook shows a data frame rendered within the Data Explorer. To the right, you'll see several visualization formats for the data. From top to bottom, you can view the data in tabular format, a bar plot, a scatter plot, an area plot, a network diagram, a hierarchy diagram, and a line plot.

![A display of the Data Explorer](https://cldup.com/ZRhONURThl.png)

Each visualization type gives you the options to modify the category, filters, color palette, and more for each visualization type.

#### Which visualization option should I use?

With so many visualization options available in nteract, which should you use? Here's a handy guide to help you choose.

- If you're planning on publishing your notebook or exporting it to a static format, you should use matplotlib or any other language-specific plotting tool.
- If you already know what kind of data visualization you want to create and visualization format you want to use and you're familiar with Plotly, you should use the Plotly transform.
- If you're looking to use the visualizations for exploratory purposes, don't have any custom needs for your visualization, and are sharing your notebooks with others using nteract, you should use the Data Explorer.

repository: nteract/docs

gha_language: HTML

repo_name: nteract/docs


text:
# ctci 📗

My solutions to coding problems from Gayle McDowell's CTCI.

## Python

It's recommend you use a venv, to prevent running into issues with black.

Inside the Python folder, run:

`python -m venv env`

And then activate the virtual environment. I've included a PowerShell script for Windows users.

A Makefile provides a wrapper for some common commands you may want to run, such as `make format` and `make run`.

## C++

TODO

repository: michaelfromyeg/ctci

branch_name: refs/heads/main

repo_name: michaelfromyeg/ctci


text:
# Ethercraft Return Extension

Shows total returns owed on ethercraft.io.  Can be used for any address.

![Example](example.png?raw=true "Example")

Also shows returns owed on each item on inventory screen.

![Item Example](example2.png?raw=true "Item Example")

### Installing
* Requires Chrome or Firefox and the [Tampermonkey](https://chrome.google.com/webstore/detail/tampermonkey/dhdgffkkebhmkfjojejmpbldmpobfkfo?hl=en) extension.
* Create a new Tampermonkey script inside the Dashboard, remove the existing code and then paste the [script code](https://raw.githubusercontent.com/jghowe/EthercraftReturnExtension/master/script) inside. After that you just need to save it and reload the ethercraft.io site.

**Donations**
If you find this extension useful, please consider donating ether (or ethercraft tokens) to help encourage further development
0x777B81225aaE4B04d5188703f1388A22B68a43B2

repository: jghowe/EthercraftReturnExtension

repo_name: jghowe/EthercraftReturnExtension