Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
Unix Operating Systems Software

Better Scheduler Than Cron? 66

Posted by Cliff from the better-than-the-reigning-champion dept.
Chanc_Gorkon asks: "I am looking for an open source job scheduler that is better than cron. I need a scheduler that has dependency checking, preemption (to stop jobs that access the same resources from running at the same time) and -full- logging. Cron e-mails are nice, but I usually end up cleaning them out with out reading. E-mailing the sysadmin -only- when a job does not get kicked off would be the preferable option. I have scoured Freshmeat and Sourceforge and have found nothing that comes close to what I am looking for. Do you guys have some options for me?"
This discussion has been archived. No new comments can be posted.

Better Scheduler Than Cron? More

Better Scheduler Than Cron?

Comments Filter:
  • A cron that supports different timezones would be good too. Why hasn't this been done?

  • launchd (Score:5, Informative)

    by Anonymous Coward on Thursday May 12, 2005 @10:50PM (#12515925)
    Have you looked at launchd, the new all-purpose task control daemon? Don't be put off by the fact that it's from Apple--it's open source, licensed under the APSL, which I believe is GPL-compatible, if you care about that kind of thing. It seems to offer many of the features you want, including task scheduling, preemption control, etc.

    Here's the manpage [apple.com]; here's a tutorial [macgeekery.com] (geared towards OS X developers, unfortunately), and here's John Siracusa's overview of launchd from Ars Technica [arstechnica.com].

  • Cron (Score:3, Interesting)

    by MrWa ( 144753 ) on Thursday May 12, 2005 @10:55PM (#12515957) Homepage
    You could start here [gentoo.org] and see if that helps.

    • Re:Cron (Score:3, Insightful)

      by BinLadenMyHero ( 688544 )
      He's right.
      Cron can do the job. Just learn how to use it, and the other unix tools.

      Many of the problems you mentioned can be solved with them.

      - For the dependencies, you can use make or a similar tool.

      - The emails are only sent if the process write to stderr.
      Write a wraper script to call the processes you want, and use the shell redirections to send some log to stderr only in the case you want.
      • Better: use a cron that doesn't send email, and just logs. Fcron is one -- it allows you to specify, on a per-job basis, whether you get email at all.
      • The key here is better and less work. I have this ONE thing I would like to do out of 40 million other things I have to do everyday. If I could get something that does this better than cron, it would save me so much time it ain't funny. I don't have the time to write the shit otherwise I would. This is one case where clsoed source has open source beat. With sufficient fundage which I don't have now, I can go out and pop a scheduler in, pop the jobs I want in it and have exactly what I had asked for. T
        • This is one case where clsoed source has open source beat. With sufficient fundage which I don't have now, I can go out and pop a scheduler in, pop the jobs I want in it and have exactly what I had asked for.

          Open source is better for me, and for many with sufficient fundage, but is not, and is not meat to be, better for everyone.

          Make for dependencies? What am I doing? Compiling?? Care to point to examples?

          Make is for managing a multi-part process with inter-dependencies. Compiling is just the most com
          • I don't mind doing some work to get things automated, but scripting 400-500 or more jobs is something I ain't got the time for and neither does anyone who does real work on a production system which is why such schedulers exist. I supposed we will end up buying XiBatch, Autosys or some other scheduler at some point, but I got to do some automation now. Currently, cron has been enough, but I can see the writing on the walls: eventually, cron is not going to be enough. It's kind of like why some people hav

    • Re:Cron (Score:3, Informative)

      by Goo.cc ( 687626 ) *

      And check out this beatiful crontab from the Gentoo forums:

      What's In Your Crontab? [gentoo.org]

  • Write some scripts (Score:5, Informative)

    by Jepler ( 6801 ) <jepler@unpythonic.net> on Thursday May 12, 2005 @10:59PM (#12515986) Homepage
    You identified some elements that could be written as simple unix scripts.

    Serializing cron jobs that access some resource? Use "lockfile". Wrap it with something that claims the necessary locks in the right order, runs the real script, and then removes all the locks.

    Only producing output in the case of an error? Write a script that saves output to a temporary file, and then cats the file if the exit value is not "success".

    These should both be pretty basic tasks that can be done in shell, python, or perl. And you can take them with you t

    • Re:Write some scripts (Score:1, Insightful)

      by Anonymous Coward
      Agreed, this kind of stuff is a half-hour project.

      Personally I use daemontools + makefiles to do complex scheduled tasks. Just write a makefile that does what you need, then create a run file that does "make -C /dir/with/makefile; sleep 2h". Daemontools will run it over and over again.

      Embellish with emailing on failure, keeping logs (multilog is good for that), whatever.

      Keep it simple, use plain text and/or the filesystem to do your config, easy stuff.
      • No it's not. How do you get cron to NOT schedule jobs on a holiday? How do you get cron to move the jobs that would have run on the holiday to another day? How do you change your schedule on a dime (emergency closing) to not run your schedule?? Cron can't do any of this easily. I need this to be understandable by more then just myself. Writing scripts and make files to do crap like this is NOT acceptable. I need more power than all of the individual tools combined can give me. I need INTEGRATION. T
    • While this is a great and doable idea, I need something operator proof. This ain't it. Also, scripts do not have the abilities I ask for. It has to WORK and it has to be EASY to setup. This is non of that. I COULD manage it, but a Operations person who has little programming experience would not be able to decipher what's going on.

  • Similiar question (Score:3, Informative)

    by Asgard ( 60200 ) * <jhmartin-s-5f7bbb@toger.us> on Thursday May 12, 2005 @11:08PM (#12516042) Homepage
    I asked a similar question here: Open Source Batch Management [slashdot.org]
  • Did we not just have an article about this? [slashdot.org]

    IIRC launchd [apple.com] does everything you're asking for (and much much more), although I'm not sure if anyone's ported it to linux yet (knowing /., that's probably an unstated requirement).
    • No, it does not. Launchd does NOT allow me to setup a chain of events that if one in the chain fails, stops and alerts an operator or sysadmin. Launchd is simplye a replacement for cron, init.d and watchdog. Also, launchd JUST came out and unelss I am using a Mac, and I ain't, I would not even attempt to use it on my production server.

  • You need to write some scripts (Score:4, Insightful)

    by StarWynd ( 751816 ) on Thursday May 12, 2005 @11:16PM (#12516099)

    The features you're talking about are really beyond the range of cron or any other "scheduler." The reason cron exists is to allow you to run jobs at a scheduled time. That's it. If you need to ensure that different cron jobs don't conflict with each other, that's a sign that those jobs need to be combined into a single script. A simple shell script wrapper around the jobs can take care of the logic you need. Just call that wrapper from cron and you should be good.

    • There are MANY close source schedulers that do exactly as I laid out but they cost mega bucks. CA makes one as do many others. I guess my Ask Slashdot was totally worthless because no one obviously works in a production environment that has a operator watching things. I guess schedulers are not sexy enough.
      • Or none of us want to write a scheduler just so that you can avoid paying "mega bucks".

        A lot of us do know what schedulers do but here's some advice that you won't have to pay "mega bucks" for.

        1. Your ask slashdot article didn't say what you were going to use it for, so all the suggestions to write scripts are perfectly reasonable. Your comments like "Writing scripts and make files to do crap like this is NOT acceptable" don't help your cause. You didn't tell give us enough information up front to help yo
        • Is it my fault you don't know what a scheduler is? No. You call my responses childish? The answers saying use cron in this way are equallly childish as well. If youare actualy a developer who's capable of writing something like a scheduler, then take this down.....I don't care what the hell you think...I have every right to say what I think and ask you what I want. Commercial programmers have to respond to their customers and if open source ever wants to be more successful, it will have to respond to w
          • You asked a question and got some answers. Most of those answers weren't very helpful for you. That much is agreed.
            You seem to be saying that anyone who couldn't give you a perfect answer should not say anything at all. Every answer you received was someone going out of their way to offer you the best information that they had. If it wasn't good enough, you just say "Thanks, but that won't be suitable because...".
            You're trying to make people wish they hadn't helped you at all, and that ruins this resou
  • ...borders on a production control system, a la autosys or control-m. I don't know what's available for free, but perhaps googling for a PCS rather than "cron replacement" would lead you in the right direction.
    • Possibly. Here's one of the closed source ones we looked at:

      http://www.taricon.com/batchfaq.html [taricon.com] (XiBatch)

      There's nothing open source that comes close to this. A scheduler like this is very common in Mainframe environments. I very rarely see anything like our mainframe one on UNIX. This one comes close. Production control cnan happen, but I want something to help set this up. I ain't got the time to setup all these little perl, bash , Python and etc crap to help me do the job schedule.
    • Hate to answer you again, but yes, your correct. Autosys is something we looked at as well. There's nothing I have found thus far that even comes CLOSE to autosys and is open source. Never heard of Control-m. Will have to check it out.
  • Dude, just use the Task Scheduler [iopus.com].

  • Options (Score:4, Informative)

    by tpv ( 155309 ) on Friday May 13, 2005 @01:27AM (#12516823) Homepage
    • Looked at ALL of these and many are not what I am asking for. I am asking for a batch job scheduler. Quartz only works for Java programs is what I gather. Most of the others are for cluster scheduling. Thanks for doing what I already DID.
      • Apart from the fact that it might not be open source (I can't find an actual license anywhere), I think Torque/Maui is exactly what you want.

        Yes it can manage across a cluster, but I'm pretty sure you can run it with a single (local) node in the cluster and all jobs with run locally. I haven't tried that, but in theory...

        Have a look at the msub command [clusterresources.com]

        The syntax is rather weird, but it seems to have everything you wanted. Except I can't quite work out whether it does dependency management. It seems

        • This sounds a little more promising. Maybe I dismissed quartz too soon. I;ll look at these sometime today and see what it looks like. I have heard of torque and I thought it MIGHT be open source and work. This is the kind of answer I was hoping for. It may not work out, but at least it's enough to get me thinking. It's a better response then the slashdotter just use cron in this way answer.

  • Have you looked at Cfengine?http://www.iu.hio.no/cfengine/ [iu.hio.no]
    I believe this can be relevant for you.

  • Yet Another /. Oddity (Score:4, Insightful)

    by kenp2002 ( 545495 ) on Friday May 13, 2005 @11:33AM (#12520205) Homepage Journal
    WTF IS WRONG WITH YOU PEOPLE!

    What is it with 80% of Slashdot readers? A person asks a question, "What other colors can I paint my bathroom besides white." 80% of the responses run along the lines, "There's nothing wrong with the color white, you just need to coordinate better with the color white." Do recent Slashdot readers and posters not understand how to answer a question?

    For Gods sake the guy asks for alternatives to Cron, not a fucking explanation on how Cron can be used to fulfill the criteria. It never ceases to amaze me on the behavior of Slashdot posters.

    "I am looking for an open source job scheduler that is better than cron." You'll note it doesn't read, "How do I use Cron to do X." First we had to blast people with RTFA, RTFP, RTFM, RTSYW (Read the shit you write) and any other obscure reference we can invent. This is clearly a case of RTFP.

    Honestly who here would be happy walking into a restaurant and ordering a steak only to get a piece of chicken and an 8 minute lecture by the waitress explaining that you just need to season the chicken properly rather then eating steak. When I order a steak I don't want a dissertation by the obviously vegan-hippie waitress on the horrors of animal slaughtering and the need to release cattle into the wild because they're the "Whales of the Great Plains." I want the bitch to ask me if I want it burnt, tough like leather, or crawling away to escape!

    How about going to an auto dealership and asking to look at trucks only to be lead to a compact 2 door Yugo with a trailer attached with bungie cords and masking tape! Fuck just answer the poor guy's question.

    --- / END RANT ---

    Here at my current client's location we built a home-brewed scheduler using PERL and POSTGRES. Jobs can be scheduled with a full range of options including every X minutes, hours, and so on along with LAST DAY OF MONTH. The development time was only about 3 weeks if you can devote that kind of time otherwise several people (Who apparently do know how to answer a question) have posted some great links on some alternatives.
    • When you order a steak, you aren't asking for the waitress to lecture you or suggest alternatives, so you have a right to call her vegan/hippie.

      When you say "What's good today? I want something that tastes like steak, but is spicy..." then the waitress is answering your question properly when she says "We can put enough spice on the steak to burn your mouth off." She doesn't deserve several dozen rants (also common on Slashdot) about how you said you didn't want steak!

      Now, one more thing before I'm done

    • Re:Yet Another /. Oddity (Score:2, Interesting)

      by rp ( 29053 )
      I think you have a serious misunderstanding about the medium.

      Like Usenet, this is a discussion forum, not a service call center. If you want straight answers to straight questions, *pay* for that right. And as someone else writes, for god's sake don't turn to a geek forum. If you want discussion on what the problem really is and how to best approach it, that;s when you go too Slashdot or some such place, but don't expect to get a direct answer to your question.
      In fact, I go to restaurants with a similar
    • You have hit the proverbial nail on the head. Open sourcce needs to grow out of the it's good enough for me it's good enough to run a multi million dollar company/college/whatever stage. Let's face it, cron sucks. Because writing schedulers are not sexy, it's gets put way down on the list if it's even on it. If you want me to use your crap, then you better give me the crap I NEED! I need something better then cron. If I write it, I will make sure to keep the source to myself and my company since cron
  • omnitpotent 'monit' can be configured than way ... and so many others ;-) http://www.tildeslash.com/monit/ [tildeslash.com]

Slashdot Top Deals

Never test for an error condition you don't know how to handle. -- Steinbach

Close