Forgot your password?
typodupeerror
Hardware IT Technology

96 Processors Under Your Desktop 350

Posted by Hemos
from the build-your-own-small-cluster dept.
Roland Piquepaille writes "A small Santa Clara-based company, Orion Multisystems, today unveils a new concept in computing, 'cluster workstations.' In October, you'll be able to choose between a 12-processor unit for less than $10,000 or a 96-processor system for less than $100,000. These new systems are powered by Efficeon processor from Transmeta and are running Fedora Linux version 2.6.6. Apparently, this new company has friends in the industry. You already can read articles in CNET News.com ("A renaissance for the workstation?"), the New York Times ("A PC That Packs Real Power, and All Just for Me," free registration, permanent link) and the Wall Street Journal ("Orion Sees Gold in Moribund Workstations," paid registration). The company is targeting engineers, life scientists and movie animators. It's too early to know if the company can be successful, but I would certainly have to get one of these systems under my desk. In this overview, I've picked the essential details from the three stories mentioned above."
This discussion has been archived. No new comments can be posted.

96 Processors Under Your Desktop

Comments Filter:
  • Cooling? (Score:5, Interesting)

    by justinmc (710870) on Monday August 30, 2004 @08:42AM (#10107898)
    Any ideas?
  • Dual 2.5GHZ (Score:5, Funny)

    by Ziwcam (766621) on Monday August 30, 2004 @08:42AM (#10107901)
    Great... by october my brand-new machine will be hopelessly out-of-date. I knew it would happen, but had no idea they'd usurp me by 94 processors.
    • Re:Dual 2.5GHZ (Score:3, Informative)

      by AKAImBatman (238306)
      I wouldn't worry too much about it. Even a single processor still tends to outclass the remaining parts in your computer, performance-wise. If you got a machine with 96 processors, you probably wouldn't notice much of a performance difference. i.e. This is only helpful to people who run heavily threaded, CPU intensive applications. Examples include:

      - CAD Modeling and testing of areodynamic vehicles
      - Modeling of Oil Wells
      - Searching for Extraterrestrial life
      - Solving very complex math issues
      - Running realis
      • Re:Dual 2.5GHZ (Score:3, Informative)

        by esarjeant (100503)
        While it's certainly cheaper to pickup 20 Dell PC's for $500 each, an integrated 12-way workstation may signify the beginning of a new desktop computing standard.

        When the IBM AT first came out, $10k was the ballpark for what was a single processor at a few mhz. Now we have a dozen procs running at a few ghz in a federated workstation environment.

        The application of this should not be understated. While SETI might seem like fools gold, the proliferation of this kind of computing horsepower could dramaticall
        • Re:Dual 2.5GHZ (Score:4, Interesting)

          by AKAImBatman (238306) <akaimbatman@NosPAm.gmail.com> on Monday August 30, 2004 @11:38AM (#10109098) Homepage Journal
          While it's certainly cheaper to pickup 20 Dell PC's for $500 each, an integrated 12-way workstation may signify the beginning of a new desktop computing standard.

          I'm certainly not disagreeing with you, but my point to the original poster is that he doesn't need to worry about this in the here and now.

          Personally, I don't see this sort of design becoming standard in a Personal Computer any time soon. Too much horsepower for a single user who simply needs more torque. However, I *do* see such designs leading into concepts like a "house computer" where the ability to multitask is more important than raw performance. Just imagine if you could install one computer for ~$2000, and have enough system resources to provide a desktop to a small office building (not to mention your entire extended family).

          Such a computer would not only provide a thin client desktop, but also handle multimedia capabilities like PVR, watching movies/TV from the internet, streaming radio stations and purchased music to anywhere in the house, interfacing with digital cameras/camcorders via Bluetooth, etc. It's even possible that such a machine could control aspects of your home via X.10, but I wouldn't count on that being a common use for quite a long time.
      • Re:Dual 2.5GHZ (Score:3, Interesting)

        by halfelven (207781)
        Also keep in mind that the Efficeon machine is only a cluster, hence it works well for (like you pointed out) heavy-duty algorithms that are easy to compute on clusters.
        Some algorithms cannot be computed efficiently on clusters; for those, you rather need a single-image supercomputer, such as the SGI Altix. Unfortunately, many of the examples you provided fall into this category. :-)
  • For a moment... (Score:3, Interesting)

    by jkrise (535370) on Monday August 30, 2004 @08:42AM (#10107902) Journal
    I thought '96 processors under your desktop! That would be the Pentium at 133MHz!

    Seriously, why 96? Why not 64 or 128?

    -
  • by slutdot (207042) on Monday August 30, 2004 @08:43AM (#10107905)
    Apparently, this new company has friends in the industry.

    Apparently Slashdot is one of them
  • yeah but (Score:4, Funny)

    by MalaclypseTheYounger (726934) on Monday August 30, 2004 @08:43AM (#10107907) Journal
    Can I run Doom 3 on it in maximum resolution mode?

    • by schtum (166052) on Monday August 30, 2004 @08:59AM (#10108027)
      If you're having lag problems, I feel bad for you son.
      I got 96 processors and you got one!"
    • Re:yeah but (Score:5, Funny)

      by WormholeFiend (674934) on Monday August 30, 2004 @10:18AM (#10108521)
      Can I run Doom 3 on it in maximum resolution mode?

      probably, but the main character is still doesnt have enough processing power out-of-the-box to use both a gun and the flashlight simultaneously...
  • strange (Score:2, Insightful)

    by ZenBased (593709)
    why not get a huge server where more users can benefit from the processing powers? and what kind of videocard does this baby pack? that must give some great doom3 performance :)
    • Re:strange (Score:5, Informative)

      by isaac (2852) on Monday August 30, 2004 @08:53AM (#10107979)
      why not get a huge server where more users can benefit from the processing powers?

      You could use these systems as such servers. The idea, though, is that these might be cheap enough to allocate to individuals.

      and what kind of videocard does this baby pack?

      No video card. These are just render/compute clusters in a box.

      I'm impressed at the claimed 220W peak power consumption of the 12-node box, but wonder what kind of real computing performance it provides.

      -Isaac

    • Re:strange (Score:3, Interesting)

      by Bazzargh (39195)
      "why not get a huge server..."

      Reading the article, the answer appears to be: politics. eg this from IDC: "There are probably plenty of engineers in the world who would love to have their own cluster so they don't have to wait for the machines in the lab"

      you what now? If its about lack of compute power on the network - usually something your project/dept contributes money to - then this comment can only mean those people who have enough money in their budget to go it alone. Most likely these people won't w
  • by Cyberax (705495) on Monday August 30, 2004 @08:44AM (#10107915)
    It'd be cool to have 12 high-end AMD processors instead of relatively slow Transmeta CPUs in this workstation. But I guess their total disspated heat will melt computer case :(
  • Sounds nice, but (Score:5, Interesting)

    by grunt107 (739510) on Monday August 30, 2004 @08:47AM (#10107932)
    One thing perplexes me:
    Chips on the same board communicate using Gigabit Ethernet, while board-to-board communication takes place on 10 Gigabit Ethernet.

    Wouldn't same board communication be more frequent, hence needing the faster connection?
    Better yet, why not 10GBe for both?
    • Re:Sounds nice, but (Score:5, Informative)

      by Vo0k (760020) on Monday August 30, 2004 @08:57AM (#10108010) Journal
      More frequent yes. But there are more parts within one board, so each of them separately needs less bandwidth than all of them taken together. So, 1G carefully engineered/switched (so each part has 1G bandwidth, not 1G shared between all) is quite sufficient. But then, say, 100 parts need 1G bandwidth between each other and 100M bandwidth to the other board, each. Makes 10G of throughput between boards easily.
    • by Xocet_00 (635069)
      If I had to guess I'd say that the on-board communication would be switched, such that the chips can talk one to one at 1Gbit.

      For chips on different boards to talk though they would need to squeeze their traffic down the same line as all the other chips trying to talk board to board. Hence the higher bandwidth?

      Just a guess.
    • by Dutch_Cap (532453) on Monday August 30, 2004 @09:03AM (#10108050)

      "Wouldn't same board communication be more frequent, hence needing the faster connection?"

      I guess it depends how you look at things. On the same board you have one processor talking to one other processor. Between boards, however, you have up to twelve processors talking to up to twelve other processors. So to me it makes sense to me to have more bandwidth between boards than internally on a board.

    • Re:Sounds nice, but (Score:4, Interesting)

      by volsung (378) <stan@mtrr.org> on Monday August 30, 2004 @09:08AM (#10108077)
      There is probably one ethernet link between boards, so you need more bandwidth on it to allow multiple nodes on the same board to communicate off-board simultaneously. Since there are 12 nodes per board, 10 Gbps is almost enough to handle theoretical worst case.

      Dunno why not 10 Gbps everywhere. If you maxed out the 400 MHz Hypertransport bus on the Efficeon, you could push out 1.5 GB/sec, which is just over 10 Gbps. I wonder how much that costs...

  • Colin Hunter... (Score:5, Interesting)

    by mantera (685223) on Monday August 30, 2004 @08:49AM (#10107951)

    I really admire this guy; although the ventures he took part in haven't gone anywhere financially, they were pretty cool. Transmeta, OQO, and now this! Go Colin Hunter!
    • Re:Colin Hunter... (Score:5, Interesting)

      by Proc6 (518858) on Monday August 30, 2004 @10:00AM (#10108412)
      Yea it's a pretty ingenious business philosophy.

      Start a company that sells CPUs. When profits are failing, start 2 more companies that can be big customers for the first company. When all 3 fail he can start another company that is built on Transmeta clusters and OQO handhelds.

      It's like floating checks around banks, but with venture capital.

  • Man..... (Score:2, Funny)

    by bhaynes (777260)
    With the power requirements on this thing, the case will be half PSU. I can see the warning on the case now 'Do not place in carpeted areas.' I bet the electro-static discharge would make you sterile faster than the speed of rubbing socks.
  • by Provocateur (133110) on Monday August 30, 2004 @08:51AM (#10107962) Homepage
    I'd like to see what the employees do with them after hours...

    "Gotta work late today, honey. Oh, sorry, is this the Pizza place? Could you please hold one sec?"

    "Gotta work late again today, honey..."

  • by Eclypser (618863) on Monday August 30, 2004 @08:51AM (#10107964)
    This time we don't have to imagine what a cluster would be like. It's already in the box!
  • by Cryogenes (324121) on Monday August 30, 2004 @08:52AM (#10107971)
    That'll be $9999, please.
  • Fedora 2.6.6? (Score:5, Insightful)

    by Halo- (175936) on Monday August 30, 2004 @08:53AM (#10107983)
    I hate being this guy, but this is a big pet peeve.

    Fedora currently is either Core 1 or Core 2. 2.6.6 is a kernel version number.

    Kernel version != Distribution

    Saying "Fedora 2.6.6" is like saying a car is a "Ford 2.4 liter".

  • Whee! (Score:5, Interesting)

    by thephotoman (791574) on Monday August 30, 2004 @08:53AM (#10107987) Journal
    96 processors running Fedora? I want one!

    Actually, I would be willing to bet that the university I'm at could use a few of these things. After all, we've got undergrads doing BLAST database work, just to teach them about it. Having been through that hell myself, it'd be a lot easier if you didn't have to have a cluster to do the work by computer. For those who don't know, BLAST is a genetic sequencing database that allows for comparison with an extracted gene (retrived through polymerase chain reaction) with a known, sequenced gene in their database.
    • Re:Whee! (Score:3, Interesting)

      by Wudbaer (48473)
      For those who don't know, BLAST is a genetic sequencing database that allows for comparison with an extracted gene (retrived through polymerase chain reaction) with a known, sequenced gene in their database.

      Nitpicking, but BLAST is not a database, it is a set of programs/algorithms for searching genomic databases (for more info [nih.gov]). But indeed such a machine should be ideal for doing BLAST searches.
    • Re:Whee! (Score:4, Informative)

      by rice0067 (220981) on Monday August 30, 2004 @10:18AM (#10108518)
      BLAST doesnt take that long anymore.. well at least not for some things.
      We use it all the time to compare our DNA products to all known Gnomes. It takes like 30 seconds.
      (300 BP search against the whole library takes less than 40 seconds. using http://www.ncbi.nlm.nih.gov/BLAST/)
      Maybe its much longer for other things ?

      • Re:Whee! (Score:3, Insightful)

        by 0bjectiv3 (216391)
        We use it all the time to compare our DNA products to all known Gnomes. It takes like 30 seconds.
        Clearly, you don't know enough Gnomes [wikipedia.org].
  • Seems Very steep (Score:5, Insightful)

    by Crashmarik (635988) on Monday August 30, 2004 @08:55AM (#10107995)
    The pricing seems quite steep. 800/cpu for 12 configuration, 1000+/cpu for the 96 cpu configuration. I can see why they have friends in the industry the prospect of selling 10 to 100 times the equipment per seat must have marketing departments salivating.

    If your'e going to spend that kind of money though theres alllready solutions that will provide that level of processing cheaper.

    There is also the utilization isssue, programming tasks hardly require 96 processors except on compile and link. You don't need 96 processors to wait for a keystroke. The same holds true in applications. You don't need cpu's waiting for a user to decide what to do.
    • Re:Seems Very steep (Score:5, Interesting)

      by timeOday (582209) on Monday August 30, 2004 @09:13AM (#10108102)
      I'm sure you're right, these aren't going to sell in huge numbers. As for the utilization issue, though, I don't think it's for programmers. Like the summary says, think animators - nothing is easier to paralellize than rendering frames in an animation.

      Personally, I think most of these will still end up as servers for groups of people instead of individual "workstations." But the logistics of a normal 100-workstation cluster are pretty bad - a large server room, enormous air conditioning unit, a massive power supply, and lots of cabling to be done. This new thing can probably share an existing server room with other computers.

      Granted, it's probably just a bit smaller and more power efficient than previous "blade" servers, but maybe presenting it as something brand new is a good marketing angle.

    • by AtomicBomb (173897) on Monday August 30, 2004 @09:43AM (#10108293) Homepage
      The CNet articles explains why the 12-way version is cheaper on a per CPU basis. The chips of the smaller 12CPU version are all mounted on single board and connected with 1Gb ethernet... While the 96-way version connects eigth 12-way CPU board with 10Gb ethernet... The high speed communication may make the 96-way system more expensive to start with.

      There is also the utilization isssue, programming tasks hardly require 96 processors except on compile and link.

      However, computer users are more than just programmers and/or IT people... Many scientific applications and animations require parallel computing... Basically, the more the better for them. They can use up any resources you throw to them. To them, the $800/CPU pricetag is not that expensive... A Sun 8-CPU machine costs them way more than $10k... A dual Xeon Dell machine with 8GB RAM/ 800GB HDD cost more than $7200...
    • I dunno.

      The new SGI Altix machines are running A LOT more per CPU than these things. I was given a budgetary quote of more than 5X this per CPU. Sure the Xeons have more grunt than the Transmeta CPU's, but is it worth 5X? Also, for applications where space is an issue, this may fit the bill very nicely.

  • by cybergrue (696844) on Monday August 30, 2004 @09:00AM (#10108031)
    OK, apparently, someone took the "Imagine a beowulf cluster of these" a bit too seriously.
    Way to go!
  • SCSI, SATA or what?
    • Re:Mass storage I/O? (Score:5, Informative)

      by Zocalo (252965) on Monday August 30, 2004 @09:13AM (#10108105) Homepage
      To save you having to RTFA and check out the product descriptions [orionmulti.com], the say "1 to 96 high performance 2.5" disk drives, 30-80GB capacity, 7.8TB max capacity on deskside". You can probably infer from the 2.5" that they are using notebook harddrives which are most likely EIDE - at least I haven't seen any SATA ones yet, although they can't be too far away.

      You should probably check out the product description anyway though; there are some quite interesting hardware design decisions in there!

      • 2.5" drives are toys anyway, SATA or otherwise. I'd have been more impressed with a fibre channel card in the thing, and leaving it driveless (for the 96cpu version, for the 12cpu, making enough room for 1TB [4 drives] of 3.5" scsi would have been ideal).

        Wow, a parenthetical in a parenthetical... god I hate monday mornings.
  • gentoo (Score:5, Funny)

    by LousyPhreak (550591) <lousyphreak@Nospam.gmx.at> on Monday August 30, 2004 @09:03AM (#10108047)
    now thats a system i'd like to install gentoo on :)
    • Re:gentoo (Score:3, Interesting)

      by shfted! (600189)
      Probably not, actually. Most programs being assembled with make never have 96 possible concurrent builds... it's usually a couple dozen at most, and quite often just a handful. Also remember that almost everything depends on something else that must be built first. Building Gentoo is actually quite linear. So for the major components of Gentoo, such as glibc and your basic windowing system libraries, you probably wouldn't notice a massive increase in speed. You might notice some benefit when compiling many
    • Re:gentoo (Score:3, Funny)

      by JamesKPolk (13313)
      Isn't it about the only workstation that can install Gentoo in a reasonable amount of time?
  • by keiferb (267153) on Monday August 30, 2004 @09:03AM (#10108048) Homepage
    I've got well over a hundred in the box under my desk. Unfortunately, it's just that. A box of over 100 CPUs, mostly Pentiums/Pentium IIs.
  • by Anonymous Coward
    Now we just need to see a company come up with a complete LAN solution where every machine acts as a SETI@Home type client in a cluster, giving SMB's a supercomputer that works while it's various workstations comprising it are idling.
  • by wsapplegate (210233) <wsapplegate@myrealbox.com> on Monday August 30, 2004 @09:17AM (#10108130) Homepage

    Warning ! Warning ! Warning ! Warning ! Warning !

    Attention, a public service announcement follows : do not read the "overview" touted by Mr. Piquepaille. This person constantly spams Slashdot [slashdot.org], trying to get traffic to his site on Radio Userland (which I'm not linking to, for obvious reasons). Do NOT go to his overview, you're only giving traffic to a spammer. See these [slashdot.org] recurring [slashdot.org] complaints [slashdot.org], for instance. Not to mention he steals the images he puts on his blog and sometimes also spews bullshit [slashdot.org] for lack of knowing better. This must stop. In any way, do not fall for the spam, and do not provide him any more traffic. Please also warn fellow readers when you see one of his self-serving posts.

    And now, a personal message (warning : verbal abuse in foreign language follows) : Roland, tu nous les brises. Va te pendre, hé Ducon !

    [disclaimer : I'm not commenting on whether the subject is interesting, or not. But the kind of astroturfing the submitter engages in regularly is just wrong]

    Warning ! Warning ! Warning ! Warning ! Warning !

    • Can the editors PLEASE STOP POSTING PRESS RELEASE COPY?!?!

      If you're going to post a story announcing a product or discovery, at least link to a weblog or site that actually has a little commentary on the subject, or the original site itself.

      Roland "Fuckyfacey" Piquepaille is neither of these.

      Thanks.
  • Reliability (Score:4, Insightful)

    by lachlan76 (770870) on Monday August 30, 2004 @09:17AM (#10108132)
    This wouldn't be as reliable as having 96/48/24/12 computers with 1/2/4/8 processors each, which would be important for things like movie animation.

    And besides for movies, we already know to just fit as many Opterons in a rack as possible. What advantage does this have (except for cost)?
    • Being able to have one of these in a standard office that doesn't need a server room setup is the major factor, as I see it. It means that the company saves on racks, rackspace, the cost of electricity (which is going up as you may have noticed by the price of oil being over $40 per barrel), administrator costs (his salary alone will be worth it) and air conditioning for the server room.

      Think of a small CG effects movie company (say around 5 to 10 employees): They want to be able to render their CG movie f
  • by Anonymous Coward on Monday August 30, 2004 @09:19AM (#10108144)
    How many stories a week does this frigging guy get into Slashdot?

    His business plan:
    1) Sell Ad Space on "News" Website
    2) Shovel In Content From Online Articles
    3) Submit To Slashdot Daily
    4) Tout "Slashdot Coverage" To Advertises
    5) Profit!

    And looking at his site, it works fine and dandy indeed.

    Then again, is he just doing a service to us?
  • That means, not so many applications will benefit from it. In fact, I'd say that 90% of desktop apps will run better on a 2 GHz single-cpu computer than on this one. In fact, a 2 or 4 CPU (non-cluster) computer would offer better performance for a larger number of apps, than any cluster. Software that takes advantage of clustered configurations has to be specifically written for them.

    Oracle RAC is one, but I can't think of any other popular title that would, expecially not for the desktop.
    • Re:It's a cluster (Score:3, Informative)

      by lachlan76 (770870)
      Oracle RAC is one, but I can't think of any other popular title that would, expecially not for the desktop

      Apache. It's on this computer I'm typing on now. Over 300 threads. I THINK it might get just a bit faster.
  • by scovetta (632629) on Monday August 30, 2004 @09:32AM (#10108217) Homepage
    100k for 96 processors? Figure you can get a barebones system with 256 MB ram for around $250. That's $24k for the boxes, a 96-port switch, and some good clustering software.

    Where's the rest of the cost coming from?

    I mean it's cool, but if I had $100k that absolutely needed to be spent, I'd get a Viper or something instead of a big server.
    <ducks>
  • from the summary:
    Orion's machines are designed like supercomputer clusters, which use many electronic brains to gang-tackle tough problems.

    Good ol' WSJ, that hardcore tech rag. Next they'll tell me the brains march off little soldiers to various parts of Computer Land to give orders and bring back messages.
  • by hattig (47930) on Monday August 30, 2004 @09:39AM (#10108257) Journal
    First, I see that Transmeta are finally making 1.5GHz Efficeons, which is a good sign, they looked to be stuck at 1GHz for so long.

    This merely looks like 12 computers on a single motherboard with a GigE switch connecting them together. Each computer is highly optimised of course, just a processor, memory, support chipset (GigE, IDE).

    I do have to wonder how it compares with something similar made with Opterons or Pentium-Ms. Opteron has the advantage of being able to do SMP so the per-system processing power would be much higher, each board could have 4 low power 2GHz Opterons which will probably be close to the 12 Efficeons in terms of computing power and power consumption.

    But still, this is a cool system. I wonder how fast it can do a kernel compile?
  • Everyone misses the point of these. A supercomputer under your desk is pretty pointless to most folk. No neon lights, so the boss won't want one to show off. Real workers most likely value decent single thread performance more than many CPUs running many threads miserably.

    For the tasks that it could do well, performance will be stunted by miserable disk performance. You can fit 96 CPUs under your desk, but they all share a 4500RPM notebook HD. Heh, yeah. Does Fedora Core have a countdown timer for measurin
  • If I remember correct, the transmeta uses code-morphing to emulate a X86. Would it be possible to remove/alter this step and create a compile option on GCC which makes code run "natively"? would this lead to noticeable improvements??
  • Would make a great platform for either a Web Server or database (or both). Imagine trying to slashdot a Web site running on one of these puppies. Would also be handy for doing load testing, and large scale compiles (re-build the universe for a large software project).
  • by theManInTheYellowHat (451261) on Monday August 30, 2004 @10:12AM (#10108491)
    Or lack there of. I was imagining one of those SGI deskside Onyx servers when I read the post but these are just wide full towers that pack 96 processors. Quite nice. I am sure the movie industry is all over these babys. The 12 unit is around the size of a Sun pizza box.

    How do the Transmeta CPU's do in fp computations? That is obviously the metric to note. I wonder how long it would take to render a movie? Is the USB USB 2? No firewire though.
  • Better solution... (Score:3, Interesting)

    by JollyFinn (267972) on Monday August 30, 2004 @10:15AM (#10108502)
    Get quad opteron. It should get about better performance for same price as the other but without need for clustering, for the small system. For bigger system, you could use myrinet and dual opterons. Oh what the heck. It costs 600-800$ per processor to build a rack of Athlon64 based cluster with Gb ethernet. So this effineon based cluster would be beaten with system costing less than their solution. So it is beaten from two different solutions.
    A) Getting single image opteron system if communicationlatencies are important.
    B) Getting cluster of AMD64 if price/performance was important.
    ONLY thing they bring is density of A for system type of B while costing more than A.
  • by Lazy Jones (8403) on Monday August 30, 2004 @10:39AM (#10108636) Homepage Journal
    In only a couple of months' time, you'll be able to put 8 Opteron cores (4 dual core CPUs) in a desktop-size case - and this is a rather reliable information. It is also very likely that similarly sized boxes with 8 CPU sockets (and thus possibly 16 cores) will appear next year: infoworld.com article [infoworld.com].
  • That thing is noisy! (Score:3, Informative)

    by lokedhs (672255) on Monday August 30, 2004 @12:11PM (#10109371)
    Do you really want one under your desk? Looking at the specs gives the sound level at "5.0 bels", which is 50 decibel. Not a pleasant work environment.

    In the end, why put this thing under your desk? Just leave it in the server room and enjoy the quiet.

Man is the best computer we can put aboard a spacecraft ... and the only one that can be mass produced with unskilled labor. -- Wernher von Braun

Working...